SD 160 2»* 



SS 010 172 



AOTHOS 
TITLE 



INSTITOTIDN 

SPONS AGENCY 

PUB DATE 
CONTRACT 
NOTE 



Love, Jchn j And Otters 

A Process l^aluation of PrGject Develcpaect al 
Continuity: Final Hspcit cf the PDC f#asitility 
Stuay, 1974-1977. 

High/Sccpe Eaucationa 1 Besearch Founaation ^ 
Ypsilanti, Mich. 

DevelcpEent 
Research and 



Office of Child 
Early ChxiahoQd 
Apr 78 

HEH-105-75»1114 ^■ 
6Bp, I For related docuffieiitSy 
163'171, and PS 010 173-176; 
the public reports gefieratea 



(DBEIi) g Washington, 
ivaluaticn Eranch, 



D^C. 



S€i ED laa 715, PS 010 
This ieries includes all 
by this studj 



EDRS PRICE 
DE SCBIPTORS 



IDENTIFIERS 



^!F-$0,83 HC-S3.50 Plus Postage, 

^Demonstration Programs; *larly Chlldhocd Education; 
ETaluaticn Criteria! E^aluaticn Methods; Measureaent 
Instruments; Program Costsi Program Descriptions; 
^Program Development ; ^Program If f ectiveniss; 
^Program Ivalua tic n 

^Developmental Continuity; ^Project DevelcFaental 
Continuity; Project Head Stait 



ABSTRACT 

This is the final report of the 3-year feasibility 
phase of a proiected 7-year longitudinal evaluation of Project 
Developmental Continuity (PDC) , a Head Start deacnstraticn program 
aimed at proviainq educational and de^elcEiental ccntinyity between 
children's Head Start and primary school esperlinces. chaEter I gives 
an overview of the PDC program and discussei the ccmponents, 
purposes, problems and iimitations of the evaluation* Chapter 2 
focuses on the program impact study coiponents cf the preliilnary 
evaluation and discusses site selecticE, lea surement instrument 
selection and refinement, data collection, and sample equivalence and 
attrition^ Chapter III focuses on the pEcgraa ifl|lementation precess 
study components of the evaluation, triefly sumaarizing planning year 
activities, planning year mcnitoring, ccnsclida tion of process 
evaluation tasks, design and testing of implementatlcn asiessment 
methods, establishment cf criteria to aeasure iiplementation, 
development of the Implementation Rating Instruient (IRI) , planning 
of data collection methoas, the fear II liplementaticn Eepott and 
Year III Implementation Study finaings, atd coit fitudy aspects of the 
implementation process study. Chapter IV gives conclusicrs and 
implications of the study, A summary cf aeasurei used in the 
evaluation ana the contents of the evaluation reports are appended. 
(BH) 
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In the surmier of 1974 HEW's Office of Child Development (now the 
Administration for Children^ Youth and Families) bftgan a new demonstration 
program aimed at promoting greater continuity between the preschool and 
elementary school experiences of Head Start. chM dren . This effort, named 
Project Developmental Continuity (PDC) , incorporated a major program 
evaluation as welK For three years the High/Scope Educational Research 
Foundation worked with its subcontractor. Development Associates , to 
provide data that would aid the Administration for Children, Youth and 
Families (ACYF) In Its efforts to design and Implement effective early 
childhood education programs, 

w 

This final report of the first phase of the evaluation of PDC 
summarizes our analysis of the feasibility of conducting a longitudinal 
study of PDC and provides a preliminary examination of factors affecting 
program implementation. A variety of audiences may find this work useful: 

9 National policymakers who must identify the best possible 

mix of programs for carrying out legislative Intent In providing 
comprehensive educational programs for children and their 
fami 1 les. 

# National and regional program administrators who must decide 
where and how to install local projects and then provide 
adequate control and technical assistance In helping projects 
use their funds more effectively. 

m Local Head Starts school and project staff who dally face the 
realities of Implementing demonstration programs. 

m The "child development research community which Is constantly 
saekrng more effective ways to help children fully develop 
thel r potent ial . • ' ^ 

In a summary such as this it Is impossible to respond completely to each 
group's information needs or to presen.t every fact we have learned about 
PDC. Instead, this report highlights the procedures and major findings 
obtained over the four^year program evaluation and briefly discusses their 
implications. More detailed Information Is presented In a series of interim 
reports. These reports, listed In Appendix B, are available from the High/Scope 
Founda t Ion . 
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PDC PROGRAM AND EVALUATION OVERVIEW 

Project Deve lopmenta ] Continuity was launched in the summer of 197^ 
with two overriding purposes, as described in the program guidelines: 

m To assure continuity of experiences for children from 
preschool through the early primary years by stimulating 
cognitive, language, social -emot lonal and physical development 
and thereby promoting educational gains for children through 
the development of social competence. 

9 To develop models for developmental continuity that can be 
implemented on a wide scale In Head Start and other child 
developm,ent programs and school systems. 

The first purpose sets the basic rationale for the program and emphasizes 
the comprehensive nature of the expected effects. The second purpose 
clearly establishes PDC as a national demonstration program:^ 



ProGram Overview 

The PDC demonstration program Is part of a major effort by the 
Administration for Children, Youth and Families to explore the value and 
feasibility of mtroducing variations on the basic Head Start theme. In 
1972 the Head Start "Improvement and Innovation" effort was announced. In 
the Head Start Newsletter for August/September 1972, Head Start Dl rector, ^ 
Jame"Robrnson, descrTbed the "1 and 1" effort as "the first substantial 
and really serious effort ever mandated to Improve the quality of Head Start 
programs." The exper imenta 1 , demonstration programs Instituted as part of 
this effort have Included, In addition to Developmental Continuity, the Home 
Start Demonstration Project, the Child and Family Resource Program, Home 
Start Training Centers, and the Early and Periodic Screening, Diagnosis 
and Treacment Program. .These programs follow In the tradition of Planned 
Variation Head Start, Follow Through, Health Start, Parent Child Development 
Centers and Parent Child Centers. 

Although Developmental Continuity shares many features of these other 
Head Start programs, It represents the first attempt by ACYF to establish^ 
a program that mandates coordination with the public schools. This coordination 
was designed to establish a consistency of approach and a continuity of 



eKperienGe that will enhance the social competence of chlluren as they 
go from the Head Start years through third grade. Although preschool s 
and publJc schools have traditionally not worked tdgether. Project 
DevelopmentaKCont Infuty aimed tc establish such a relationship. Typically, 
public schools have not been concerned with comprehensive services and 

arent involvement In the way Head Start programs have. Through PDC, 
b^houls and Head Start 'centers have the opportunity to work together to 
improve the continuity between them. 

Two PDC program models were instituted to provide alternattve vvays 
of establishing the administrative structure for continuity. in the 
Preschool-School Linkages approachi administratively separate Head Start 
and elementary programs are brought together by the device of a PDC Council, 
whose membership includes teachers , .parents , administrators from both 
organizations, and community representatives. In the Early Childhood 
Schools approach, Head Start and elementary programs are combined both 
administratively by the Council and physically In the same building, thus 
creating a new institution. Regardless of the approach, a qualitatively 
different program is expected to emerge as a result of the Head Startle 1 ementa ry 
school cooperation. 

Continuity is expected to be established in two contexts: that of the 
individual child and that of the school structure. in the first context, 
continuity means, for example, that a child should not have to have his or 
her personal nature and needs rediscovered each year as he or she moves from 
one grade to the next; instead the child should become a more and more fully 
recognized member of the school "family'' as time passes. In the context 
of school structure, continuity implies cooperatfve pursuit of common goals, 
and this involves articulation of philosophies and methods in all the 
various areas of school enterprise. It Is expected that structural continuity 
will contribute directly to xont inui ty in the attention given to Individual 
chi Idren. 

In the summer of 1973 the Huron Institute of Cambridge, Massachusetts 
was awarded a contract to develop a concept paper on program options for 
providing continuity of services. During. the winter of 1973'7^ the 
Administration for Children, Youth and Families assembled a panel to review 
the various options that were outlined In the Huron report. The present 
conceptualization of Project Developmental Continuity was decided upon, and 
during late winter and early spring 197^, a draft of guidelines for the 
Initial planning year was drawn up with the assistance of Huron Institute 
staff. At the same time, Requests for Proposals were being developed by 
ACYF for providing technical assistance to the programs and for conducting 
a national evaluation. 

Potential PDC sites were first Identified because administrators, parents, 
teachers and community people at those sites all expressed Interest in 
participating In the program. The selection process involved several steps. 



First each regional ACYF office^ and the Indian and Migrant Program Division 
asked a number of Head Start grantees within their regiorl to complate a 
questionnaire to determine the feasibility of Implementing a PDC program 
at their site. Based on responses to this survey ,\the four sites deemed 
by each region to be mosit suitable were recommended to the national ACYF 
office. A review panel then selected two of these four sites to submit, 
proposals. Staff from the^ national and regional ACYF offices reviewed 
these proposals and visited each site to meet with grantee staff, Head Start 
and pufelic school teachers , parents and Head Start and school administrators 
and review project plans. Since PDC was intended to extend beyond Head Start 
into the early-elementary grades, U.S. 'Office of Education staff were Involved 
In the selection process, along with state education agency staff. Through 
this process one site was selected from each region, axcept Regions 111 
and Viil, which each had two. Two additional sites were selected to represent 
the Indian and Migrant Program Division of ACYF Four sites were designated 
Bilingual Btcultural Demonstration Projects.- 

This selection process, resulted In local settings for PDC that are 
extremely diverse, ranging from the large urban populations served by the 
Utah, Iowa* and Washington projects, to more jubi/rban settings In 
Connecticut and Maryland, and finally to smaller, rural communities sn 
Texas, Florida and Arizona. The ethnic and cultural compositions of 
these communities ar also diverse, Including, for example, Navajos, Hispanic& 
Blacks and Appalachians. 

Operation of'the program began In 197^ at 15 sites and the entire first 
year of program operatlnn was designated a planning year for local projects. 
Staff were-hired, component area task forces were appointed, and detaijed 
plans for actual implementation were Initiated. " ^ 

^During Year M, 1975-76, l4 sites (the New York site had withdrawn), 
comprising a total of 42 .Head Start centers and elementary schools, began 
to Implement thejr plans. Program Year 111 (1976'77) was officially 
designated ^s the 'Mmfpl ementat ion year" in the original project design, and 
by Year Ml programs !we re expected to be ,f ul 1y Mmplemented and operational. 
After Year J I the Nevl Jersey Site withdrew, resulting in the current 13 sites. 
During the third program year ACYF decided to continue funding the 13 projects 
beyond the third program year to permit program 'operations to continue through 
1980-81. ^ ^ ^ . 



^The ACYF regional offices are located in Boston (Region 1^, New York City 
(Region II), Ph 1 1 ade 1 ph 1 a - (Reg 1 on II I ) , Atlanta (Region IV), Chicago 
(Region V) , Daltlas (Region VI), Kansas City (Region Vll), Denver (Region 
VIII) ; San Francisco (Region IX) , and Seattle (Region X) . 



^The original 15 sites were located in Arizona, California, Colorado, 
Connecticut, Florida, Georgia, Iowa, Maryland, Klchlgan, New Jersey, 
New York, Texas, Utah, Washington, and West Vf rginia. 
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The Role of the National Office . . _ 

— . - - -- ^ 

Washington-based staff in ACYF's Program Development and Innovation 
Division are responsible for administering the national program. They 
have maintained contact with sites through te lephone cal 1 s , site visits, 
written cormun i cat ions , and national workshops. They also helped the 
sites by contracting with outside groups to provide training and technical 
assistance. w_ ^ 

During the planning year and the program start-up year, the Huron 
institute served as the T&TA contractor. As part of this effort, a staff 
of field spaciaHsts (each one working with one or two PDC programs) made ^ 
several visits to the PDC sites. The PDC.T&TA philosophy Implemented by 
Huron institute staff was one that stressed facilitating pa rt i cl pat i^on 
among all groups involved In PDC at a site, emphasizing all areas of the 
guidelines', and exposing sites. to as many alternatives as possible before - 
making final prografrniat i c decisions, Huron Institute also worked with 
national ACYF staff in planning the national workshops. 

An initlar planning meeting held In September 197^ involved local 
project staff, regional ACYF staff,' T&TA staff, and representatives from 
state education agencies. At workshops held In January and May of the 
Planning Year, evaluation contractor staff also participated wi th these 
other groups. Two national workshops were held during each of the next two 
years, and In May 1977 a PDC meeting was -held In conjunction with a national 
conference on ''Chi Idren , FamM 1 es , and Continuity." These workshops 
provided a continuing forum for discussion of Implementation issues (each 
workshop typically had One or two themes such as parent involvement or 
multicultural education), review of guidelinss and f und I ng I ssues , and 
communication about evaluation matters. In addition to the content workshops 
and guidance from nationarand regional program staff, a. beneficial feature 
of these meetings was the opportunity for staffs from the diverse projects 
to share information, ideas and experiences. 

At the beginning ^of the third program year, Pacific Consultants of 
Washington, D.C, was selected as the TfiTA contractor. One field specialist 
was selected to work with each PDC site. In addition to providing assistance 
to sites and organising two PDC workshops, Pacific Consultants published three. 
Issues of a PDC Information Bui letin, Irf which conference summaries, articles 
related to PDC Implementation issues, and bibl iographic Information on * 
materials relevant to PDC were disseminated to the sites. 

Program Guidel Inas * - 

Over the course'-of PDC, two sets of program guideUnes have been published 
by ACYF, In spring 197^ Guidel Ines for a Planning Year was dl strlbuted to 
prospective sites to serve as a guide for preparing Initial proposals. This 
set of gU del Ines wap revised In/September 197^, and supplemented by ''Program 
Lettars'f^ Issued periodically from the national office. During the second 
year (November 1975) a new set of guidelines, PDC Implementation Year Gulderines , 
war published, describing basic elements that must be present In each PDC' 
project. ^' 



The guidelines have consistently outlined requirements and suggestions 
in the following seven component areas* As such, they have not only 
provided a framework for the Impl ementft ion Study , but have served to 
define the dimensions that a comprehensive Impact |tudy should possess: 

• Administra t ion s admf n i st rat 1 ve coord i nat 1 on between,, 
and within Head Start and the elementary school (s) ; 

^ p Educatjont coordination of curriculum approaches 

^ and educational goals; /. 

• Tram jjrg t preeervlce and^ inservice teacher staff and 
parent training in program^related areas; 

m Developmental supp ort services : comprehensive services 
^niedlcal, nutr 1 1 lonal , and social)" to children and 
- f ami lies; 

^ ^ ^ r ^ ^ 

m Parent I nvol vement : parent participation In policymaking, 
home-school activities, and classroom visits or volunteering; 

m Services for the handica pped: services for handicapped 
children and children with learning disabilities; 

m Bilingual b I cultural and myl t icul tur4>1 education : programs 
for bilingual blcultural or multicultural children. 

As the Implementation findings discussed in this report will show, 
there Is considerable diversity among the 13 remaining PDC prog^ms in their 
response to the various guideline requirements. 



Purpo se Qf__the PDC Evaluation 

The major purpose of the PDC evaluation Is tq aid the Administration 
for Children, Youth and^amllies in Its efforts to design effective programs 
for children. As Figure 1 illustrates, the evaluation was planned in two 
phases: the first to determine the feas i b M i ty of conducting a Igngitudlnal 
study of PDC (197^^1977) and the second to carry out that study as children 
progress through the third grade (1977^1981). 

The first phase had tvvo major components^^an Impact Study and an 
Implementation Study. The Impact Study was charged with seven major tasks 
during the first three years: 



m Identify the most suitable comparison Head Start centers 
and elementary schools at each site* 



Figure 1 

Cohorts Participating in PDC Impact Study 



Grade Level 



Head Start 



Feasibi 1 i ty Study 
Phase 



P rojected Long I 
tudinal Study 



197^-75 Planning Year 



1975-76 



1976-77 



.1977-78 

1978- 79 

1979- 80 

1980- 81^ 



Cohort 1 
(fall s 
spring) 



Cohot-t 2 
(fall 
spri ng) 



Cohort 2 
(spring 
only) 



Cohort 2 
(spring 
only) 



Cohort 2 
'(spring 
only) 



Cohort 2 
(spring 
only) 
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• Select and/of' develop measures for assessing program 
inipact on cf^ildren, parents, teachers, staff. 

I / .? • 

• Pilot test the measures to determine their reliability, ^ 
validity, sensitivity to changes relevance to "social 
competence,!- ease of administrations and suitability for 
use In higher grades. - ^ 

m Determine, on the basis of demographic data and test scores^ 
whpther the PDC and comparison groups are really comparable. 

• Analyze attrition data at the PDC and comparison schools to 
determine (a) the sample size needed at each site and then 
(b) whether attrition during 1976^77 was consls'tent with 
project/i ons . 

• Provide a preliminary analysis of program Impact through the 
Head Start year (1976-77). ^ . 

• Search for poss Ibje rel at ionsh i ps, between Implementation and 
^ Impact* ^ 

Although the Impact Study has been llmlced thus far (as explained 
below) to the study of program impact on children^ it is expected that 
the projected longitudinal study will conceptualize Impact more broadly 
to include parent participation and attitudes^ teacher attitudes and 
work styles, and the organ I zat ionaU cl Imates of the schools, In addition 
to Impact on children's social competence. ■ 

The Implementation Study was designed ta describe and analyze the 
processes that led to the measured consequences of the program. There 
were five basi c. purposes of the study- / 

• bescribe the nature of the PDC. treatment at each site^ 
including descrl pt Ions' of program costs . 

• Describe and analyze national patterns in the Implementation 

of PDC I y . 

• Assess the extent to whicJi each program Implemented the basic 
PDC gu i del I nes . 

: 0, Understand the factors and /events that have shaped program 
implementarlon* / ' ' 

• Assess similarities and differences in experiences provided 
for chl 1 dren in .the PDC and comparison schools. 

Major program activities of the first phase of the PDC evaluation are 
described in greater detail in Chapters II and I IK 
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Efforts to describe and analyze program processes began during 
the PDC planning year (197^-75) with the preparation of site case 
studies. During the following year the design for the full Implementation 
Study was finalized and pilot data were collected at five sites to 
evaluate the applicability of the interview forms and the procedures . 
for rating implementation levels. On the basis of the analysis of the 
pilot data, modifications In procedures were made and. a major Ins'trument 
for assessl.ng" iraplementation, the Implementation Rating Instrument (IRI), 
was final ized. , 

In the third year this' Instrument was applied to the interview data 
and other documentation from nine sites to provide a comprehensive assess- 
ment of implementati'on actiylt'ies in PDC. Three additional sites were 
included in various documentation act I vlt^les but did not receive the 
systematic Implementation ratings. At the thirteenth site, a Navajo 
program In Arizona, a case history approach to assessing both implementat lo 
and impact was taken. 

Throughout the three-year study, evaluation efforts have been about 
equally divided between process evaluation tasks (case studies, monitoring. 
Implementation assessment and cost analysis) and impact tasks. The pie 
chart presented on the following page illustrates the division of labor and 
provides a perspective on the breadth of the study. 



LI ml tat Ions on the Evaluation 



the first phase of the- PDC evaluation has been constrained by 
the inability to employ several key data col lectlOn instruments. This , . 
cl rcumstance exists, because a delay of months In the forms clearance 
process at the Office of Management and Budget (OMB) made it impossible 
to conduct some of the planned evaluation activities. The Impact Study, 
originally conceptualized as a broad assessment of impacts on parents, , 
teachers and the institutions as well as on fih I 1 dren was limited to a study 
of impact solely on children; the Implementation Study was restricted to a 
portion of Its potential sources of Information, was not able to include all 
sites, and was prevented f rom, eKSmining comparison school programs. 

Two key Impact Study instruments had been developed, but were not cleared 
in time to be used. One was a Teacher Survey developed to assess teachers 
attitudes and perceptions of PDC and their own Institutions. The second was 
a Parent Survey designed to obtain information on the extent of parent 
participation In and attitudes toward the program. 



, PDC EVALUATION 
RELATIVE EMPHASIS GIVEN TO EVALUATION COMPONENTS 
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The extensive interview forms needed for collecting detailed 
information for the Implementation Study were also affected. Without 
0MB clearance, however. It was still possible to conduct interviews 
with PDC staff most knowledgeable about each component since the interview 
questions for a particular component were answered by a single individual 
at a site; it was necessary, however, to limit this data collection to 
nine sites,- In the Implementation Study design we had also planned to 
interview a sample of parents and teachers at each site to obtain supplementary 
and corroborating information; nine' such interviews were conducted at one 
site, but could not be carried out at the other sites because of 0MB 
regulations, , , . 

Another major gap created by the forms clearance problem involved 
the interview forms that were designed to col lect compa ratsle programmatic 
data from comparison school ' pri ncl pa 1 s and Head Start directors. This 
Information would haye permi tted some assessment of the sjml lar I ty of ' 
treatments in the PDC and comparison settings. I 

In spite of these dif f icul ties , almost all the evaluation purpbses 
listed on pages 5 and 7 have been addressed. Under the Impact Study the 
feasibility of a long-term study has been adequately assessed, and the 
Implementation Study has provided Important descriptive and explanatory 
information about the process of PDC ' Implementation. ^ These accomplishments 
are reviewed in some detail In the following chapters. 



^OMB regulations stipulate that clearance is not required If there are nine 
or fewer respondents for a particular form; if the total number of 
respondents (considering al 1 sites and all timepoin.ts) Is ten or more^ 
official 0MB clearance must be obtained. Since .the Interview related to 
a particular prograjTi component rdquired on ly one respondent per sitej it 
was possible to conduct a complete set of Interviews at nine sites. 



REVIEW OF IMPACT STUDY ACTIVITIES 

The first year of the Impact; Study was a year of preparation durfng 
which there were two major concerns* locating suitable Head Start centers 
and elementary schools in each site to/serve as ^'conipar I sons" for the 
PDC Head Start center and school or schoolSj and selecting measures that 
would be appropriate for assessing, the impact of Project Developmental . 
Continuity. Fol lowing these 6ffortSs Years I I and II L were. devoted to 
assessing the adequacy of the. measu.res , conf i rmi ng the suitability of the 
samples and, in Year II i^; assess Ing^program Impact through the Head Start 
year. Each of these is discussed [n turn in this chapter. 



• ? -;\ Sel ect i on of Compa r I son Centers^ and Schools 

By early summeT 1975 the evaluation had collected sufficient information 
from each of the PDC sites to provide Initial estimates of sample sjzjps 
needed for- Cohort 2 in fall.1976 and to assess the comparabi U ty of PDC 
and compari soa Head Start centers and. elementary schools. Beginning in 
December 197^ i sites were asked to identify the schools and centers that 
would be 11 kely locations for the comparVson group. At a national PDC 
workshopMn. January 1975^ forms were dls^tributed to all s I tes request i ng 
i nformat Ibn;:Qn the-ethn i c i ty , native language, sex-^ family sTie> mother-s 
education, preschool attendance, and free lunch eligibility of children in 
the PDC centers and schools and in the candidate comparison sites. 

/ For each center and school , percentages wera corriputed for the various 
sample characteristics, and initial Judgments of comparabl 1 i ty were made 
on the basis of these center- or school-level statistics. In a report 
submitted to ACYF In June 1975, recommendatipns were made for each site 
regarding the feasibility of continuing the Impact Study. Several special 
conditions were noted in that report! 

• The Arizona project obtained special permission for testing 
^ Navajo children. 

• Several sites reported difficulties In providing distinct 
POC and comparison programs at the Head Start level. 



• At the Georgia site special efforts were made to enlist 
the cooperation of school board In a neighboring county 
since PDC comprised the only school in its district. 
When this effort failed, a special design was racon;mended 
using cross^sectional data from children in the PDC school 
in 1975^76 against which to compare Cohort 2 children as 
thjy progressed through the grades* ^ 

• The prospect of cross^distrlct busing at several sites 
emerged as .a 'potential threat to maintaining samples for 
the evaluation* 

Not until Years M and Ml could group comparability be assessed 
on the basis of data CQllected on children. Although the groups turned 
out to be highly comparable (see p* 21), in June 1975 it appeared that the 
overall prospects for continuing the evaluation looked very good.at only 
four of the sites, that three sites had problems both with comparability 
of the comparison centers or schools and with potential attrition> two 
sites had problems only with group comparabi 1 ! ty , and the data from four 
sites led us to be seriousl y concerned wi th attrition; the New York site 
had by then announced its withdrawal from the^ program and the Georgia prograrr 
represented a specfal case with no comparison program* Al though cond it i ons 
at many of the sites could not be described as ideal for instituting a 
longitudinal study, a number of recommendations were made to Improve the 
prospects* Our report to ACYF essential lyl'recommended collecting additionaj' 
information during Year l!| based^on actual chlld^ level demographic and 
test-score data^ in order to make a more rigorous analysis of group 
comparability* .With respect to attrition concernsj in thft second year of 
the study we used rnore refined methods of estimating the percentages of 
children who would remain in the sample at each site through third grade. 



Selection of Measures 



In January 1975 evaluation staff began to review literature In 
preparation for recommending measures for assessing program Impact on 
children, parents and teachers and for assessing Institutional change* 
This effort culminated In the selection of 12 tests or subtests, a child 
interview, a teacher rating scale, a tester rating scale, and a classroom 
observatfoh system to be used, across all sites and six additional measures that 
could be elected by sites who wished to have other areas represented i n their 
evaluation.^ In addition, the development of teacher and parent surveys was 
recommended , « 



iThe original intention of the test battery was to be responsive botji to 
the generalized goals of PDC (through a ''basic battery") and to spe^cific 
goals of Individual sites (through si te-specific measures that could be 
selected by particular sites). Although the PDC sites were sympathetic with 
turn intention, only four elected any of the site-specific measures; when the 
ps^hometric qualities of these measures later provad to be unaGCeptable, 
and ACYF decided to discontinue the option of site-specific measures, we 
heard no complaints from the sites. 

/ ^ ■ . 12 

v' ■ 22 / . 



Selection Process 



Before beginning the selection process, a number of criteria 
were establ i shed by which all candidate measures could be Judged, First, 
SIX general guidelines vmre articulated (some of which had been specified 
In the evaluation RFP) i (I) no new measures would be developed; (2) adapta- 
tions or rrradif Icatlons, of existing measures would be undertaken to make 
them more appropriate to this study; (3) Instruments for the *'basic 
battery" would be evaluated In terms of site objectives; (k) testing time 
per child would be limited to a reasonable amount (two hours, divided into 
two or three sessions, was allowed for the pilot testing with the understanding 
that the battery would be streamlined for subsequent data collection periods); 
CS) the battery wo.uld be as simple and parsimonious as possible in order to 
increase accuracy of data collection under the complicated field conditions; 
and (6) single items withUi Instruments would be Individually Interpretable , 
• assuming that straightforward InterpretabI I I ty at the. Item level would avoid 
some of ^the problems of Inferring theoretical processes.^ 

After applying these general guidelines, five specific criteria were 
used to assess the suitability of each measure reviewed. In order of , 
importance, the measures were required to- (I) appear to measure national - 
or local objectivesr (2) be appropriate to the chl 1 dren ' s ages , ability ' 
levels, ethnicity and bilingual bicultural status; (3) be practical to i 
administer; (k) have been used In other major evaluations; and (5) demonstrate 
good psychometric characteristics. 

The potential instruments that passed an Initial screening along the^. 
lines of these-guldennes and criteria were rated on ]k factors and tabled 
in Interim Report II. These l4 factors, incorporating elements of the 
above criteria plus additional concu/rns, were grouped into four sets: 

• Practical considferat Ions (available for use by fall 1975; 
appropriate for trained paraprofess I ona 1 s ; test format 
appropriate for PDC age groups; scoring procedures appropriate 
for data processing; reasonable testing time for young children), ^ 



^As the analysis of. measures occurred at successive stages of the evaluation, 
this last guideline assumed less and less importance. At best its application 
depends upon the nature of the particular instrument; though a partlcujar 
ob&ervatibn category may have meaning as an Item of behavior, or an Item 
assessing f actua 1 . 1 nformat i on might be important fpr its own sake, most of 
the measures employed in the evaluation are meani ngful * only if a group of 
items (factor or scale) behives consistently and can reasonably be said 
to relate to a psychological dimension of Interest to the program planners. 
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• Psychometric qualities (adequate construct and/or 
predictive validity^ adequate test stability and 
Internal consistency; absence of cultural and/or SIS 
bias; representativeness of standardization sample}. 

• Relevance to PDC (spans age range of DC population, 
*4-8; Spanish adaptation available; relevant to PDC goals; 

likely to 'demonstrate program effects) . 

• Past use (used In previous national evaluations or 
larqe-scale studies). 

One of the major goals in 1975 was to establish measures of children's 
"social competence. This goal was In large measure attributable to the 
influence of Edward ZIgler, who as Director of the Office of Child 
Development in 1972 described Head Start as hopingi 

. , ,to bi'ing about gT&ater soaiat aompetenQe in disadvantaged 
ahildr&ni By aoaial aomp&tena& is meant an individual's 
everyday effBotiveness in dealing with his enviromtent, . *hi^s 
ability to maBt&r appropi^iate fajmal aonaepts^ to pepfoim well 
/ in sahool^ to atay out of tToyble with the Im^^ and to relate 
well to adultB and other ahildren (quoted by Anderson and 
MeB&iak^^ 1974^ p. 283). 

The> initial conceptual I gat (on of the task of assessing children's 
social competence drew heaviiy upon the discussion of 29 competencies 
b<f Anderson and Messiek (197^) and the reviews of the Rand Corporation for 
its design of a national evaluation of Head Start (Raizen & Bobrow, 197h) ; 
the Identification of social ^emotional behaviors was especiaj ly Influenced by 
the work of White and Watts (I97j) andOgllvie and Shapiro (19731 ^ 

The final conceptualization of social competence was perhaps closest 
to the framework outlined by Anderson and Hessick, prliriarMy because their 
competencies appeared to be rfiore Inclusive and thus closer to PDC's concern 
with develop I ng each child's "everyday effectiveness In dealing with his 
environment and responsibilities In school and life"; the Rand approach, on 
the other h^nd* placed greater emphasis on the^chlld's effectiveness in the 
"role of pupil," Areas of expected Impact on teachers and parents were also 
developed In cdhsultation with members of the evaluation's advisory panel and 
staff at ACYF, ' 

In an attempt to take program goals more specifically Into consideration 
as^ the measures were being selected, a "PDC r^rogram Goals Questionnaire" 
was developed In the spring of 1975. This questionnaire contained statements 
describing 25 child social competencies, l8 teacher or staff goals, and 10 
goals for parents.^ The bccasion of a national workshop held in May 1975 was 
u^ed to explain what I nformat 1 on we wanted and how each PDC project could 
complete the questlonnai^re nn such a w#y that It would represent the opinions 
of the local program. Four goals for children were r/ited as "most Important 
by five or more sites^ 

.24 



m general language use 



• problem«so1 ving skills , 

9 sel f--di recting; being^bla to influence the 
outcome of events 

• health and nutrition, 

A few sites (I.e,, less than five but more than one) listed the following ' 
as "most important--^ learning how to learns realistic sel f ^apprai sal*, 
recognition of feelings In self and others^ sensitivity to and understanding 
of othei^, social problem-solving, positive school att i tude , gross motor 
skills, and perceptual motor skills. The predomf nance of so many goals 
that are inadequately assessed by available tests was ah important factor 
In leading us to develop observational and rating procedures for tapping 
child characteristics in the social-emotional domain. 

The '*most Important" goal for staff listed by five or jrore sftes wai - 
that teachers become better able to Individualize Instruction, The ''most 
important" goal for parents was that they have more Input Into deci si ons 
about the total school program for all children. 

Final Selection of the Pilot Year Battery 

The literature review was conducted attempting to take Into' consideration 
the definitions of social competence, the various criteria listed above, 
and program goals* Sources included other evaluation studies of Head Start 
and related programs, published instrument reviews* journal publications 
and consultant opinfonsp The measures reviewed were d I scussed.iand evaluated 
in Interim Report M, along with our recommendations for the battery, The 
Instrument review was organized according to broad program goal areas : 
children's social-emotlonal development; psychomotor development; health and 
nutrition; cognitive and language development; Impact on teachers and 
parents; and bilingual bicultural Impact goals. For%ach of these areas 
the report described\the relevant theoretical constructs, the potential 
measures, and reasons for selecting or rejecting particular measures. 

These recommendations were reviewed by the PDC Evaluation Advisory 
Panel, outside consultants, and ACYF evaluation and program staff* Hodifl- 
cations were made in the recommended battery, and almost Immediately the 
task of preparing for data collection began. All measures except the 
Bilingual Syntax Measure had to be transited Into Spanish, flour measures were 
translated' Into Navajo,^ the classroom observation procedure iwas finalized, 

- ' ■ ■ / ■ ■ ■ 

^Tape-recorded oral translations were made of four measures/ in the psychomotor 
domain by natia/e Navajo speakers. Analysis of the fall pretest data showed 
all to have questionable reliability. I t was therefore reconmended that \^ 
testing" be discontinued at the Arizona site and that a case history approach 
to assessing both Implementation and impact be instltut^^d. 
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copyright clearances for the published tests were secured, test booklets 
and administrat jon manuals were prepared, testers were recruited and hired 
from the PDC communities, and on September 1375 eight trainers and kk 
testers convened In Michigan for eight days of intensive training. 

1 The measures that were administered in that first fall testing are 
rndicated in the first column of Table 1. ^(Because of the large number 
of measures that were potentially worthy of Inclusion In the PDC battery 
and the limits on the amount of testing four-year-old children can tolerate, 
not all measures were administered tu'all children J Parent and teacher 
que's tl onna i res were developed but not administered at that time; they were 
submitted to the Office of Management and Budget for forms clearance with 
the expectation of being able to administer them by spring 1976. 

Data Co] lection Procedures ' 

. Careful procedures were developed to enhance the quanty of the data.^ 
The procedures included (a) an organizational structure that prescribed roles 
and responsibilities for High/Scope staff, local site coordinators and the 
testers; (b) a training model that provided supportive instruction under 
a variety of conditions and careful analysis of Individual tester performance 
(c) onsite monitoring of testers under field conditions before the beginning 
of actual data collection; (d) weekly monitoring among Ipcal testing staff , 
throughout the ^^ata collection; and (e) careful check-In procedures by 
High/Scope data processing staff and communication of problems to the site 
coordinators. 

Over the four testing periods from fall 1975 to spring 1977 there was 
excellent retention of local testers. Of the 3^ testers employed In spring 
1977, 50 percent had worked with the evaluation since fall 1975. This 
considerably eased the task of training for each successive data collection. 

Similar procedures were followed during each data collection period* 
though each period also. had its special features. In spring 1976, for 
example, the child measures were adml n I s tered to chiidren in the upper grades 
at two sites to obtain estimates of their su I tab I 1 I ty ''for use beyond the 
Head Start year; in spring 1976 and spring 1977 a bilingual measure of oral 
productive language was tried out on small samples. All of these activities, 
had as a basic purpose the development , >ef I nement and validation of a set 
of instruments that would provide a meaningful assessment of children's 
social competence as it might be affected by PDC. 



^Although PDC had Its unique characteristics that affected these procedures, 
; we were fortunate to be able to draw heavily upon the ^procedures developed 

by High/Scope and Abt Associates during the course of the National Home 

Start Evaluation, 
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Child Measures During the First Phase of the PDC Evaluation 
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^Letter indicates measure was administered at that timepoint and that one of the 
following conclusions was made: \ 



■C ^ Continue to use the measure with little or no rTwdification \ 
P ^ Provisionally retain measure, attempting to correct minor problems with 
reliabilltyi validity or administration procedures / 
, R ~ Refine the measures through further developTOnt work ^ 

D ^ Discontinue the meaiure 
' - ^ fteasure not administered 

^RofMn numeral indicates number of interim report in which analyses are reported, 

^Administration of WPPSI and WISC Block Design subtests was discontinued 
once tjieir suitability as covariates was established; they were not 
intended to be repeatedly administered. ^ . 

'♦In addition to other refinements, after spring 1976 it was recomfT^nded that 
the observation system be used for assessing classrooms rather than .individual 
children. ' ^ 

.^Direct reasure. 



%te. » Each of the measures is briefly described in Appindix A, 



Assessing the Adequacy of the Heasures 

Over a two-year period, the initial battery of 17 tests, two, 
observation instruments and two rating scales has been winnowed to 
seven tests ^ two observat i on instruments and two rating scales (see Table 1). 
In the process* much has been learned about the interrelationships of 
these measures and the structure of the battery as a whole, and the prospects 
for a useful evaluation of child outcomes, at least in the short run, seem 
good at this time. Further, the Spanish- language versions of the instruments 
appear to be about equally suitable for their intended sample* 

At each of the four timepoints when tests, observat ions , and ratings 
were completed, extensive data analysis was undertaken in order to assess 
the. acceptabi 1 1 ty of the measures for the purposes of the evaluation. 
Although the analyses differed in detail from time to time (for example. 
It was not until spring 1976 that test-retest stability analyses could be 
conducted), taken as a whole, they were designed to provide information on 
the following character i St ics ^ reliability, validity, sensitivity to change, 
relevance to social competence, suitability for use in the higher grades, 
and ease of administration. , Decisions made about the measures at each 
time point are summarized in Table K ■ 

Rel iab i 1 i ty 

In spring 1977 the internal consistency reliability coefficient was 
.65 or greater for all measures In both the English- and Spanish-dominant 
samples. Host measures remained constant In their reliability indices 
across the four timepoints at which they were administered during this 
evaluation. ^Changes In scoring increased the reliability of two measures, 
but the reliability of another measure has declined slightly over time. 



Valldi ty 

The validation procedures' involved determlalng the expected relationship 
of each measure with each of the others, then comparing these expectations 
with the relationships that actually appeared in the data. Under" this 
convergent-discriminant method of assessing validity, the assumption is made 
that if an instrument Is actually measuring the construct it Is intended to 
measure, the instrument will correlate highly with other measures of the 
same general construct, will correlate moderately with measures of similar 
constructs, and will not correlate at all with measures of Independent 
constructs. All the instruments ^Hamined are acceptably valid for Head Start 
children, as evidenced by the stability of their validity indices across two 
cohort^' and three timepoints. 



23 

18 



Sensitivity to Chang e 

Since the- Impact Study depends upon the PDC battery of measures = to 
datect changes that can be attributed to program differencas, three types 
of analyses based on the pilot samples' fall J975 and spring 1976 data 
were carr I ed out : 

a The correlation of each measure in the fall and In th 
spring with child age at the time of testing was 
calculated to determine the age- rel atedness of the 
measures; the correlations tended to be low, positive, 
and significant, with coefficients generally between .15 and .30. 

# The difference between children's fall scores and spring 
scores on each measure was analyzed to ascertain If the ^ 
scores increased significantly from fall to spring; all 
measures showed a significant fa 1 1 -to-spr 1 ng^ increase, 

# A regression procedure was used to determine whether the 
observed spring mean on a measura was equal to or greater 
than the score that would be expected knowing how much 
^older the chl 1 d ren .were In the^sprlng. For all measures 
except the BSM-Engllsh, the mean actua 1 score was 
significantly higher than the mean expected score, indicating 
that the tests are sensitive to change beyond that which is 
simply a function of increased age, (The. most likely factor 
to explain this additional gain is the children's Head Start 
experience/) 



Relevance to Social Competence 

Since the PDC battery was constituted with the intent of measuring 
the traits that comprise social competence, analyses were performed for 
Interim Reports IV and VII that examined the relationship of test scores 
to ad hoc criteria of social competence. The criteria were established by 
facTo^r analyzing ratings completed by each chtld*s teacher and tester, and 
then creating factor scores for each child that representrn his or her status 
on each of the '•social competence" factors. The assessments provided by 
the teachers and testers are based upon observations of each child's behavior 
irr a variety of f^mal and informal situations, and thus 1 og lea 1 1 y come close 
to representing measures of the child's "everyday effectiveness," I.e., 
social competence. 

The object of the analysis (a linear regression procedure) was to 
determine the magnitude of the relationship eKlsting between the tests 
Included In the PDC battery and the/'social competence" criteria. The more 
relevant the tests are to social competence, the stronger^ the relationship ^ 
expected. In spring 1977 all tests were found to be substantially associated 
wi^h-the collective "soci a 1 competence" criteria. Thus, these tests, 
originally selected for their theoretical relevance to social competence, 
ieem to provide measures that are empl r leal ly Relevant to social competence 
as wel 1 . . 
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Su itabMity for Use In the Higher Grades 

During the 1975-76 testing periods, approximately 25 children per 
grade (kindergarten through grade 3) were tested at the Georgia site 
as part of the cross-sectional design there. In addition, 30 third 
graders were tested In Maryland. 

Conclusions about the suftabillty of the child measures for use at 
each of these grades v/ere based on four factors: response distributions 
on the j terns of each measure, mean scores on each measure, reliability 
(interna! cons I s tency) , and validity. Based on these factors, all of the 
measures appear to be useful through grade 3, either in their present 
forms or with modifications. 



Ease of AdmJ n I stratJon 

One of the factors taken Into consideration when tests w^ere being 
reviewed for the PDC Impact Study was their general suitability for 
admlTiistrat ion by pa raprof ess 1 ona I s . in general, monitoring of testers 
during training and daja col lection indicates that the tests have not been 
difficult to administer. Tester performance Ir^proves with practice and 
administration difficulties are more apparent with new testers than with 
experienced ones. 



Assessing the Sujtablllty of the Samples 



The two chief dimensions of 'sample suHabillty examined in this study^ 
are the comparability of^the PDC .and comparison groups and sample size. Since 
the ultimate test of PDC's effedts'on children Is made by comparing their^ 
performance at some future point with a group of children who have not 
participated In PDC, it Is essential that the initial equivalence of the two 
groups along important dimensions be established. Further, these tests of 
PDC 's effects will- require that a sufficiently large sample remain by the 
end of third grade If a complete test of theHongi tudlnal impact of PDC is 
to be made. Since there are really two samples within the PDC study (children 
whose dominant language Is English and children at the bl 1 1 ngual bi cul tural 
demonstration sites whose dominant language Is Spanish), the suitabtlity 
of each sample was assessed separately. 

Similarity of PDC and Comparison Group s 

As mentioned above, the comparison sites, were I n i 1 1 a 1 1 y se lected to 
resemble the PDC Head Start centers and elementary schools along some 
important dimensions. Once data collection was underway, however , I t became 
possible to compare group means on a large number of variables for which 
child-level data were available. This was done for Cohort 1 during i975'76 
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in order to gain a sense of how IJkely it was that group comparability 
would be achieved for Cohort 2,- The results of these analyses were 
highly encouraging, but >^here problems exi sted efforts were undertaken 
to improve group comparability. In spring 1976 sites were informed of 
the nature of the PDC^compari son group ,d i fferences and they attempted 
to modify recruitment practices so as to provide a better match. The 
important comparability analyses were conducted on fall 1976 data from 
Cohort Zf For each test and r%t i ng sea 1 e and for six background 



number of siblings and mother ' s educa 1 1 oh) the assumption of PDC-compar i son 
group comparability was tested statistically using the chi^square technique 
for categorical variables and ^ tests for metric variables/ From these 
analyses v^e reached the following conclusfons In Interim Report VI: 

# At the individual site level the groups appeared similar; 
there were differences on background variables in only 

one site and very few d I fferences on any of the performance 
measures. 

# At the aggregate level the groups appeared extremely 
similar; for the' Engl ish-dominant sample^ there were no 
significant group differences on background variables and 
only one difference In test performance; for the 
Spanish-dominant sample ^ the groups differed on only 

one background variable, and there were no differences 
on any of the performance measures. 

For a' long I tud i na 1 study in which extensive attrition is expected, 
Jt is also neq^essary to show that attrition does not bias the relative 
composition of the PDC and comparison groups. Potential biasing due to 
attrition was a^sesspd in two^ v/ays : (1) the PDC and comparison children 
remaining in the sample in spring 1977 were compared on their fall 1976 
test performance and backg round characteristics (to determine whether the 
two groups are still comparable); and (2) the children remaining In each 
group In spring .1977 were compared with those who had left (to determine 
whether the rema i n i ng xh i 1 dren are representative of the original sample). 
Very few diffsrences were found In either of these analyses for either ^ 
the English- or Spanish-dominant samples. These analyses led us to conclude 
(in Interim Report VII) that the groups are^ still comparable and that each 
group still reflects the characteristics of the original sample. The fact 
that both these dimenslohs remain stable after 121 of the sample departed 
is noteworthy and encourages optimism that the processes accounting for 
attrition may be operating In ways that will not seriously bias the groups 
In the future. 



character! st Ics 
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In Interim Report VI evidence was presented to show that the sample 
sizes were likely to remain adequate for conducting analyses with all 
sites compared. Attrition has been a major concern since, with smaller 
samples, it becomes increasingly difficult for Che statistical analyses 
to separate PDC's effects from the effects of the many other factors that 
contribute to children's performance. Several procedures were used for 
estimating the attrition that will be likely at each of the PDC sites, 
initial projections made In spring 1976 were used to determine the Cohort 2 
sample size sites were expected to achieve, ^ In Interim Report VI we 
projected that, overall, about kO% of the PDC group and 361 of the comparison 
group would remain at third grade. Considering that further reductions 
in sample size will occur when certain handicapped children cannot be 
tested or when children refuse to cooperate on part icular measures , it was 
estimated that about 205 PDC and 170 comparison children would be left in 
the English-dominant sample four years from now. 

At> the request of ACYF we asked each PDC site to locate children 
from the original fall 1976 sample who are currently enrolled in PDC and 
comparison school kindergarten classes. ^ There are currently ^10 PDC and 
322 comparison children enrolled In the appropriate schools (651 retention 
in PDC and 571 In the comparison group). ^ These figures are eKtremely close 
to the numbers and percentages projected In Interim Report VI, These figures 
are encouraging in the aggregate , al though there are some sites at which the 
samples will soon diminish to a size that would make testing uneconomical. 
There are five sites with fewer than 30 children in the PDC kindergarten 
group and one has only 19 dhil^ren remaining;^ one site has only 14 children 
remaining In the comparison group. 



^t that time the Florida site seemed plagued with both small Initia-I sample 
sizes due to the size of the Head Start program and very high attrition 
rates. In spring 1976 (Interim Report 111) we recommended that consideration 
be -given to dropping the site from the child impact aspect of the eva 1 uat i on , 
The site presented additional information, however, suggesting that a tt ri t i on 
was not so great among the current Head Start population, so a special . effort 
was made early In fall 1976 to collect additional data through personal I nterviews 
on site. On the basis of this information, it was decided to include Florida 
in the evaluation; as of fall 1977, Florida now appears to have. the lowest 
rate of attrition of all the PDC sites* 

2As of November 1977. 

3The k\Q PDC children are located in 19 elementary schools across the 12^sites, 
and the 322 comparison children are spread across 37 schools, .j 

i 

newly enacted state law in Georgia changed the date by which kindergarten 
children must be five years old from December 1 to September I, forcing 16 
of last' year's Head Start chi ldren to remain in Head Start another year. 
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Assessing Program Impact ihrough the Head Start Year 



Once the ©valuation had provided evidence that the cHild measures 
ware adequate* that the PDC and comparison groups were highiy comparable^ 
and that large enough samples would remain (at least in the aggregate) 
for a lorigltudinal study, the stage was set for examining Impact through 
the Head Start year (1976-77)* Differences between the scores of PDC and 
comparison group children on each measure were subjected to analysis of 
covariance (performed separately for the English^ and the Span i sh^doml nant 
samples). Pre^test was used as a covariate and sex, ethnicity, prior 
preschool experience, and site were- Introduced as blocking factors to 
further . reduce error variance*^ 

Table 2 shows the adjusted spring scores of the PDC and comparison groups 
on each of the test and rating scale variables. Across the 30 analyses 
performed for the English- and Spanish^domfnant samples^ four were 
statistically significant (all favoring the comparison group). Although 
one might take the statistically s I gn I f I cant d i f ferences to imply that the 
corriparison group children gained more during the Head Start year^ than the 
PDC children J these differences are too small to be of any practical importance. 
Thus we conclude that the two groups remain essentially identical at the end 
of the Head Start year. 

Most planners and participants In the PDC demonstration effort expect 
the effects of Head Start^el ementary school continuity to appear only after 
the children progress Into the elementary grades. Thus these findings 
confirm the general expectation that the two groups would be comparable 
through the Head Start year. The fact that the two groups began as 
virtually equivalent groups in. fall 1976 and that they received highly 
similar Head Start eKperiences during 1976"77 means that the groups began 
kindergarten (m fall 1977) still evenly matched. This sets the' stage for 
a clear test of the developmental cont i nu 1 ty hypothes i s during the next^ 
phase of the evaluation. 



-Details of these analyses are described In Interim Report VII, Volume 3. 

^In only one case does the magnitude of the difference between group means 
exceed *25 standard deviations (see Table 2); In all cases the percent of 
variance accounted for by group membership is negligible (for the three 
d i f ferences M n the Engl 1 sh-'domi nant sample the variance accounted for by 
group membership did not exceed 2% and for the single difference in the 
Span I sh^doml nan t sample group membership accounted for 81 of the'.varlance) . 
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Relativi Stitus of PDC and Compirison Groups on Dependent Measures 
(Values Adjyitid for Extrinious Factors), Spring 1977 













PDC- 
Comparison 
Differ'fince 


- — 

Standard 
Deviation 
of the Measure 


Group Diffirenci 
as % of SO 
(shown for 
significant 

dlfffrences only) 




Adjustid Group Maans 


I Measure 


pnr 


N 

If 


ConiD 


1! 


ENGLISH-DOMINANT SAMPLE 














■ 


BSM-English 


10.23- 


470 


10.43 


411 


-.20 






WPPSI 


8,58 


m 


8,77. 


271 


-.19 


A 17 
j/ 




Verbal Fluency 


9.12 


m 


9,33 


406 ■ 








\ Verba 1 Memory- 1 


14.79 


470 


15.25 


412 


-,47 


5.43 

LJ I 


1 IP 


Verbal Meiiiory-3 


3.69 


464 


4.18 


40S 




Arm Coordination 


4,54 , 


464 


4.43 


404 


+.11 


3i4 1 




Draw-A-Child 


5.48 


4BS 


5.97 


40B 






PIPS ^ • 






L .OJ 




-.21 


1 ./u 




POCL-1 : Task Orientation 


34,47 


iM 


34.09 


392 


+ .38 


3 ,00 


\ 


POCL-2: SoQiMity 


13.54 




13J0 


292 


+ .44 


3.62 




CRS-1; Frtemlinm 


lo./y 


461 
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QQH 


+.30 














* -.13 


5.35 




CRS-3: Pemevermae 






29 23 




' +.05 


5.99 
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Ad 1 






+ 17 

'fit 


1.61 
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CRS-5: SBlf'ABBUMnoe 
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CRS-6: Resom^oe fulness 
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".29. 


2.54 


.06 
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-.37* 
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2.20 


Height 
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BSM-English 


7^82 


f 


4.05 


s 


+3,77 






; BSM-Spanish 


12,56 


52 


12.44 


. 43 


±10 

+. u 
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Verbal Fluency 


8.41 


17 


7.78 


44 


+ .63 


SI 




Verbal Memory-1 


17.34 


17 


16.90 


44 


+ .44^ 


£ 1 n 

Bjg 




'Virbil MimQry-3 


5.02 


i% 


4.53 


44 


+ .49 






• Ann Coordination 


6.88 


M 


5.87 


4S 


+1.01 


3.86 




Oraw-A-Child 


6.42 


3? 


6.40 


44 


+.02 


1 .97 


.^3 


PIPS 


2.68 ' 


17 


3.69 ' 


44 


-1,01* 


1.90 ' 


POCL-1 : Taik Dnentation 


39,37 


ih 


35,56 


42 


+3.81 


9.83 




POCL-2: miability 


13.61 


li 


13.13 


42 


+ .48 


4,07 




Wiight 
Height 


41,02 


13 


40.24 


3fl 


+.78 


5.17 




41 .95' 


53 


42.10 


S9 


-.15 


2.11 





*p.< .05; **p < .01; ***p < .001. 
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Examining PDC-s Impact Under Specif ic Conditions 



Despite the generally equivalent overall progresi of the PDC and 
comparison groups. It Is conceivable that PDC might have had a differential 
effect under certain conditions. For example, one might expect greater 
program effects at some sites than at others. ^Analyses were carried out 
to examine the possibility that PDC*s effect might have been different 
depending upon the site or upon certain characteristics of the child (seXj 
ethnicity^ and prior preschool experience). No statistically significant 
effects were found; in other^words, there is no evidence to indicate that 
POC's effects are more promtnerjt under some conditions than under others. 
Thus, through the Head Start year at least, PDC is equally effective for 
boys and girls, for all ethnic groups, for children from all sites, and for 
all children regardless of prior preschool experience. 
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REVIEW OF IMPLEMENTATION STUDY ACTIVMTIES 

The ppocess evaluation of POC was originally designed as four separate 
substudies: monitoring, case studies, imp rementat !on study , and cost 
analysis. During Year I (the planning year), evaluation activities clearly 
fell into these d I f ferent areas ^ but It became apparent that the purposes 
of the evaluation would be better served by consolidating efforts where 
It vJBs conceptually appropriate. Thus, in Years It and III all process 
evaluation activities were subsumed under the name of the Implementation 
Study. ^ This chapter describes the fJrst^year study of the PDC planning 
process as reported in the Planning Year Case Studies, describes the 
development of the complete Implementation Study in Years I I and J II , 
summarizes findings from the Implementation Study, and presents findings 
from £he collection of PDC financial data in the Cost Study, 



Planning Year Case Studies 

The case studies for the planning year documented efforts by each of 
the original 15 PDC sites to set up the organizational structure and communi-^ 
cation network necessary for such an innovative program. While dealing 
with the same basic issues, each case study is unique, reflecting the diverse 
populations, educational settings^ geographic locations, staffing patterns, 
and major program goals of each project. The planning year case studies 
are important documents. Nowhere else In this three^year evaluation effort 
do we present in such detail the inner workings of each site, as well as 
the interactions and attitudes of program personnel. The case studies 
preserve the history of PDC and its 15 original projects and describe the 
foundation upon which each program has been shaped, furnishing clues for 
understanding both the successes and failures sites have experienced. 



^The Cost Study, though conceptually part of the process evaluation, has 
been conducted and, for the most part reported, independently. 



Data Col 1 ect ion Strategy 



Data for the ]97^-^75 reports were collected in two v^eek-long visits 
to each project, one In the winter and the other In late spring of the 
planning year. Information was obtained by data collection teams, primarily 
through interviews with PDC staff, Head Start and elementary school adminis- 
trators and teachers, parents and other program participants. Information 
v^as verified through committee minutes, questionnaires, classroom observations 
and PDC proposals. 

Each site received a copy of their draft case study report to review, 
and site comments or revisions were incorporated into the year-end case 
studies. Drafts of the year-end reports were also sent to the sites for 
review, so that the final planning year case study for each site contains 
information from the draft report, an updated summary of planning activities 
through the -^end of the school year, and additional perspectives from local, 
regional and national reviewers. 



Summary oF Planning Y ear Findings 

The conclusions reached in this phase of the PDC evaluation are based 
in part on Information collected for the planning year case studies, but 
interpreted in light of subsequent data. Even by the end of the planning 
year, however, there were Important, though tentative, conclusions represented 
In the case studies that helped shape the direction the Implementation Study 
was to take. Some of these preliminary conclusions are reviewed here 
because they influenced the development of evaluation strategies for the 
Implementation Study. 

PDC programs generally focused their planning year efforts on establishin 
organizational structures " for the local program. Staff needed to be hired; 
the PDC Council had to be established; committees or task forces had to 
become functional; community agencies had to be contacted; and. most 
importantly, the communication links between Head Start centers and elementary 
school (s) had to be set up and maintained. At some sites these tasks 
were accomplished within the first few months; at other sites, It took 
the better part of the planning year to establish the organizational 
structure for the program. 

A number of factors seemed to influence sites' progress in accomplishing 
the casks outlined In the planning year guidelines. Institutional support 
was important, from both the Head Start program and the school system. 
Program model made a difference; the Early Childhood School (ECS) sites 
had a somewhat easier time establishing administrative linkages than the 
Preschool -School Linkage (PSL) sites. The experience and knowledge of 



28 



key PDC staff, especially in early childhood edgcation and administration, 
was important to smooth progress on planning year tasks. Staffing was 
also important; since there were a large number of requirements for the 
planning year in all of the program components^ sites with two or more 
full-time component coordinators were generally able to make more progress. 
By the same token, an active PDC Council contributed to the work of the 
program, by stimulating other groups as well as substantively contributing 
to planninq. 



Planning Year Honftorlng 

The basic purpose of this task was to assess (l) the extent to which 
site activities and s i te object Ives matched the model described In the 
PDC planning year guidelines^ and (2) the extent of compliance with Head 
Start performance standards. Another objective of the monitoring task 
was to determine problem areas which required attpntfon and, perhaps, 
technical assistance^ and to inform ACYF so that corrective action could 
be taken. 

As a separate and dls^ net task, monitoring was discontinued at the 
end of Year I for two reasons. First, monitoring was already carried 
out by ACYF's regional offices. Second, the monitoring reports' function 
of describing program status at a given point in time could be accomplished 
by including descriptive site reports as part of the Implementation Study 
dur i ng Years 1 1 and 111. 

Monitoring data were collected fn winter and spring of 1975 during 
case study visits through the Interviews with project staff, adml n i st rators ^ 
teachers J and parents, and through the review of project documents mentioned 
above. An interim monitoring report on each s i t6 was submitted to ACYF in . 
April 1375, and in June 1975 revised monitoring reports' were submitted to 
ACYF on 14 PDC sites for the entire planning year.^ 

While onsite, evaluation staff reviewed the information collected 
to ensure that each required planning task was included. As a group the 
site team then reviewed the planning tasks and assigned codes Indicating 
the degrer. of compliance of each task, The team members prepared a concise 
explanation for assigning the compliance code and made appropriate recom^ 
mendatlons. The reports were submitted to ACYF for review and possible 
action. 



See list of reports in Appendix B. 



The Monitoring Study producad no "findrngs" as such, although the 
systematic way in which it summarized sites' progress on each planning 
task was useful to ths case study writers. The essential findings were 
the site-level lists of tasks completed and not completed. Although the 
national program office may not have needed this information for making 
technical assistance plans, ^ several staff from the PDC projects commented 
that the process of reviewing planning tasks in such detail had been 
extremely useful to them. 



Consolidation of Process Evaluatio n Tasks 

Based on experience In Year I, it was decided that the purposes of 
the process evaluation could best be met by combining the Imp laments t i on , 
monitoring and case studies. Three cons i dera t I ons prompted this decision: 

^ Xhe Year I monitoring and Implementation studies were, 
redundant in many ways; both included observati^on of^ 
programs and assessn>ent of compliance with ACYF require- 
ments. The principal difference between the two was not 
in their data collection or analytic tasks,, but in the 
uses to which findings were to be put. The Monitoring 
Study was to be an instrument for program development; 
the Implementation Study was a component in the overall 
study of the PDC program. Thus, by combining the two 
studies the data collection burden on the sites could 
be reduced with no loss of Information. 

m Portions of the Monitoring Study were redun^dant with 
existing Head Start and PDC monl tor Ing acti;vitles. A 
major task for the Monitoring Study, the monitoring 
of program compliance with Head Start performance 
standards, was already being performed by ACYF^s regional 
offices. Duplication of this effort In the PDC evaluation 
was unnecessary. 

m The case studies, while distinct conceptually from the 
Implementation and monitoring studies, were dependent 
- upon implementation data to be Intelligible; the reasons 
for educational change could not be described adequately 

l|n fact, the frequent visits to PDC sites by staff from the technical 
assistance contractor as well as at least one visit per site by a national 
program officer provided ACYF with direct information on planning activities 
and seemed to lessen their need for monitoring reports. 
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until the change Itself had been described* Similarly, 
data on the processes of change are most efficiently, 
collected together with data on the nature of the changes. 
Thus, by combining case studies with the other two studies, 
considerable duplication of effort could be avoided. 
The preparatido^of Individual reports on each site was 
continued as part of the implementation Study , though the 
reports were turned Into straightforward descriptive 
accounts with little analysis or Interpretation. 



Design and Pilot Test of implementation Assessment Methods 

Year II (1975-1976) was devoted to designing and pilot testing the 
methods for measuring program implefnentation. During that year criteria 
were established f or' measur I ng I mp 1 ementat ion , data collection instruments 
were finalized, and a report summarizing this work was submitted. 

Before these procedures are .descr I bed , however, it is important to 
note that the PDC Implementation Study differs from other studies of the 
educatlonaT change process In several ways.^ First, much of the literature 
either describes the Implementation of curriculum elements such as a 
special reading program (e.q>, the review by Fullan & Pomfret, 1977) 
or the creation of complete /*new schools*' that are not simply redesigns 
of existing programs (e.g./Hiles, Sullivan, Gold, Silver, S Wilder, 1978). 
Although PDC is a more comprehensive Innovation than a single curriculum 
element, it Is not quite a *^new school*' either. PDC was a "frameworl< for 
innovation^' that provided general . gutdel ines sl<etchlng what the program 
might look like, but did not confine the programs to detailed requirements. 
This fact clearly affected the way we developed criteria for measuring 
Implementation, Second, PDC frequently involves several schools and 
Head Start centers, whereas much of the research on educational change has 
focused on innovative programs in single school buildings. Third, PDC 
was designed to create linkages between existing programs; the literature 
on educational change processes typically focuses on change within a system 
rather than the linkages between systems. Finally, each PDC project is 
part of a national evaluation study, and most projects have had to consider 
the needs of the evaluation when making decisions, and could not simply 
do what might have been best programmat 1 ca 1 1 y . Awareness of these features 
of the PDC demonstration program has contributed to the methodology developed 
for assessing^ program implementation. . 
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^ Establishing Criteria to Measure Implementation 

- , Any attempt to place different programs alohg a common quantitative 

^ dimension Is certain to encounter problemSj and the PDC evaluation was 
no exception. PDC was never Intended to be a comprehensive model that 
Would-be installed faithfully at each site. Instead, each ^ oca 1 project . 
was expected to develop, within certam guidennes, a program suited to 
local needs. Were PDC a -s Ingle model, systematic cri terla ^and procedures 
for implementation assessment would be relatively straightforward to 
develop. Since It Is not, care had to be taken not to impose more structure 
' on. programs ti+ian the guidelines dictated. Still, a common framework for 
observing and assessing implementation across sites was needed because 
'the PdC demonstration project was^designed to answer questions for ACYF 
regarding whether a particular type of educat lonal intervention could 
produce the desired outcomes In children. ^ 

Developing -the IRI > ^ 

The implementation Rating Instrument (iRl), developed In Years II 
and III, attempted to reconcile these conflicting demands^ by defining 
rating criteria for (a) the extent to.>/hlch sites have Implemented basic 
gt^ideline requirements and (b) participants* views of the effectiveness 
of that 'implementation, witbout specifying the precl se nateire of the 
program. Events forced some compromises in this position, but the 
basic focus of the IRI was upon the^degree to which programs Implemented ^ . 
their own particular Interpretation of the guideline requirements. 

The IRI Is. a set of I terns which allows site visit staff to system- 
atfcaJly evaluate the programmatic information that Is collected by a 
variety of mea^ns (Interviews^ records, and site documeots primarily). 
Two ty^es of scales were developed for maWng ratings of Implementations- 
objective and judgmental. 

" The IRI objective rating scales were developed by eKtracting a list 
of discrete program requirements from the guidel ines and then devising 
a set of rating scales. that couVd be u^ad to assess the extent and 
'effectiveness (as perceived by part ici t>ifnts) with which a site had imple- 
mented thej/r programmatic solution to^ tha« requirement. Through this 
process almost 350 separate 4-point, rating scales were developed that ^ 
span the seven program component areas. These scales (called ''objective" 
because their ratings were based on explicit, and often quantitative 
•^H program data) are of four basic types: those rating the presence of the 
various program attivitles at the Head Strrt and elementary school levels; 
' tTiose rating the extent of that implementation (In terms of numbers of 

children affected, classrooms involved, etc.); those assessing participants' 
percept lons'^of the effectiveness of that implementation; and finally, those 
assessirig the roles played by different groups in Implementation decisions 
and activltiesV^ ^ . 



The judgmental scales asked the site team members to reassess the 
program's implementation levels based on everything they knew or felt 
about the program. The Judgmental scales were designed to tap two of 
the same dimensions as the objective scales (the extent, and effectiveness 
of Implementation) plus an additional dimension, the ^-Intensity" or amount 
of effort and Importance accorded the subcomponent^ by project staff. 
By these procedures we hoped to address two problems^of implementation 
assessment:^ equal weighting and the obscuring of mitigating circumstances* 
Objective I terns gave equal importance to each program element; the 
judgmental allowed evaluator's '-weightings-' to Influence the ratings. 
Objective ratings made no allowance for mitigating circumstances; the 
judgmental allowed a variety of circumstances and situations to influence 
the ratings. 

Initially^j the Intention was to use both types of rating scales in 
the spring 1976 field test at five sites and, based on that test, to 
select the most useful type for inclusion In the full data collection* 
The results using the two scales (see Interim Report IV) were sufficiently 
different in the field test, however, that both scales were retained 
in the revised Implementation Rating Instrument used in Year III* 

Planning Data Collection Methods 

\lnformation on which, the ratings were based needed to be obtained 
from several sources. Nine data col lection strategies were developed 
during the first two years of the evaluation that could have been used in 
the implementation Study: 

m' Structured interviews to be conducted with PDC administrative 
and teaching staffs during site visits;- - s 

m Ethnographic (i*e*, non- I nstrumented) observations of PDC 
classes during site visits (Year II only); 

• Systematic observations of PDC classes by local testers 
using a time-sampling observation Instrument (Years 11 
and Ml); 



^Eleven separate interviews were developed: six for local personnel 
. most knowledgeable about each program component (administration, 
education , bi 1 Ingual -mul t icul tural , handicapped, parent involvement , 
developmental support services), one for parents, one for teachers, 
one for the program administrator, one specifically for bilingual 
bicultural demonstration programs, and one for the PDC Councli chairperson 
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• Parent questionnaires mailed to a random sample of PDC and 
comparison school parents as* part of the Impact Study 

l^ar I I I only) ;1 . 

• Teacher Surveys conducted with a sample of PDC and comparison 
school teachers as part of the Impact Study (Year III only)^; 

o Documents (i*e*, proposals, curriculum statements, etc.) 
collected from sites (Years l-lll)j 

m Data collected as part of the cost analysis (Years II and 
III); 

m An, optional onsite record-keepfng system to be used by PDC 
staff to record needed information on PDC meetings, training 
activities, and delivery of required health and social 
services (Year III); 

■ / . 

m A structured interview to be conducted with prmcipals of 
comparison elementary schools and directors of comparison 
Head Start programs to obtain programmatic Information on 
the comparison programs (Year lll);^ " 

Data relevant to most Implementation variables were obtained through 
the structured interviews, with the other strategies supplying auxiliary 
or verification information. Site documents, the record-keeping system, 
and Cost Study data have been, however, a primary source for certain 
highly quantitative data (for example, average monthly volunteer hours) 
which would be difficult and t ime=c6nsu/ni ng to col lect In interviews. 
The Parent Survey was to have obtained opinions from parents about the 
effectiveness with which various parent^ involvement requl rements have 
been Implemented, so data on the actua 1 Unumber and kinds of parent activities 
were obtained from the other sources. 

Year II Implementation Report 

By August of Year II, these activities culminated in a report to ACYF 
(Interim Report IV,. Volume 2) that also presented a preliminary analysis 
of factors that appeared to relate to levels of program Implementation* 
Analysis of s i tes' efforts to implement PDC revealed that a small set of 



^The Parent and Teacher Surveys could not be adml n I s tered due^ to delays, 
in obtaining forms clearance from 0MB, 

^The Interviews with comparison principals or Head Start directors could 
not be administered due to delays In obtaining forms clearance from 0MB. 
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identifiable factors seemed to repeatedly Influance" local Attempts to 
implement PDC. If anything, the analysis underscored what is so often 
emphasized In .the literatures efforts to mtroduce change in exTsting 
schoQl programs are exceedingly difficult* P^C attempted to effect 
systemic changes In both Head Start and,, elementary schools, and the 
agents of this change had to contend wl'jh (or capitalize upon) existing 
regularities within the schools and comhiun 1 1 les . 

i * ' 

11' . _ /. 

The report went on to identify 37 factors which seemed to facilitate 
or Impede Implementation progress, and firom this listj hypotheses were 
formulated which related specific factorts' to program implementation levels 
measured by, the IRh These factors and 'hypotheses formed the basis of 
the third'year implementation evaluatloni and provided the framework 
around which analysrs^f Implementation v^s* carried out* 



Year 1 I I i mg^l^mentat ion. Study 

In fall 1976 the record^keeping system--^evel6ped during Year M was 
installed by asking each site to maintain foVms for documenting develop-^ 
mental support services provided, parent classroom participation, business 
conducted' dur 1 ng PDC meetings, and the nature and frequency of training 
act 1 V i t i es V 

Between January'and March 1977 nine sites were visited^ and on the 
basis of records maintained on the above forms and interviews conducted 
with PDC staff, site teams rated levels of implementation using the IRI.^ 
The findings from Year III data collection are reported In two volumer 
in Interim Report VII. Volume 1 presents the national perspective, descrlbi 
patterns and processes of implementation acrdss sites and proposes possible 
explanations for the observed levels of Implementation* Volume 2 provides 
backup detail, with descriptive accounts of each site's program activities* 
The findings of Interim Report VII are summarrzed here. 



^As explained earlier, the number of sites Included in the complete Imple^, 
mentation Study was. In part, determined by what data col lection activities 
were legitimate in the absence of 0MB forms clearance./ In addition, one 
site (Arizona) was not Included In this aspect of the data col lect i on 
because of the special case study design that was used for that project* 
It should be pointed out, however^ that various levels of Information 
were available on all 15 sites for at least portions of the PDC implemen- 
tation perlodj and this Information was used In our analyses where it 
was considered appropriate* 



Findings Related to Patterns of Implementation Activities 

One of the TOSt St r i k I ng findings from the Implementation Study was 
Ithe wide variation in ratings across sites* When Xhls variation was 
analyzed some I nterest i ng patterns emerged; thesa patterns^ with respect 
to the major program components , are summarized here. Following this 
section we present a synopsis of the factors that seem to have influenced 
the activities and the ratings, f . ^ ' 

Educat lon > Each of the projects has developed or adopted a curriculum 
that can be appl ied f rom Head Start through thirdgrade, A few sites 
decided to purchase and adapt existing curriculum "packages," while 
others chose to develop their ov/n curricula by making major changes in 
existing Head Start or schoor curricula* Several other sites decided that 
they already had curricula that fulfilled the PDC guidelines. By these 
varying means, .almost all sites received high Implementation ratings 
In the education component. Thus^ whatever other emphases a site may have 
hadj it seems that classroom i nstruct i on was paramount. 

Bilingu al bicultural and/or multicultural education . - The sites that 
were designated as bilingual demonstration programs were much more likely 
to implement an approach to bilingual educat|on that could be classified 
as "maintenance'' program (l/e,, conscious efforts were made to maintain 
the children's mother tongue at the same time they were learning Engl i sh) ; 
the approach of most sites, however, could be characterized as either ESL 
(English as a second language) or transitional bilingual ism (in which the 
native language was used for instruction only at the Head Start level). 

Services for handicapped childj^en . Ma i nst reami ng of handicapped ■ 
children In regular classes at both Head Start and elementary school levels 
was evident at all sites. .The comprehensiveness of services for handicappe< 
children, however, seemed to be more a function of other state and local 
programs than of the efforts of PDC. 

4 Parent Involvement . There was considerable site=to-site variation 
in parent Involvement ratings. The variations reflected both differences 
In program emphases and local obstacles to achieving parent participation 
when It was actively sought. The PDC projects have been more likely to 
strive for parent participation in classroom activities than to emphasize 
parent Involvement in substantive program decision-making. Head Start 
parents were generally more likely to be involved as classroom volunteers 
than elementary parents. 
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Developmental support services . There was considerable consistency \ - 
across^ites in thl^a7ea. Most suites provided the required screening \ 
'and follow-up services to Head Start and elementary children, provldea^ 
at least some training for staff, and kept records in accordance with ; 
program guldelinei. 

Administration , Levels of Implementation in this component depended,^ 
on how well^thT~PDC Councils functioned and the extent to which Council 
members participated In program policy decision-making. Onjy five sites 
had Councils that Included all of the member groups required by the 
guidelines, and the size of the Councils ranged frdm 11 members at one 
site to 36 at another. Although the formal authority of Councils ranged 
from having decision-making powers. to being strictly advisory* the 
"advisory*' functions were often equivalent to decision-making* At most 
sites, elementary sch9ol ^ parents , teachers, and administrators were better 
represented on the POC Councils than Head Start parents, teachers and 
administrators. In one of the more interesting variations in PDC staffing 
patterns, it was found that PDC coordinators were, In some cases, res^-. 
ponsible for only one program component-coverall program administrat lon^-but 
at some sites coordinators were* respons 1 b 1 s for as many as three components. 

" '. 

Training . Training activities varied considerably across sites as 
each project attempted to meet guidelines requirements and its own needs 
in different ways. Some sjtes^recei ved high ratings for implementing 
all aspects of the t ra 1 nTng' component ; others focused their training in 
particular areas (e.g., training staff in parent Involvement) and received 
high ratings only in those areas. 

Findings Related to Determinants of implementation 

As mentioned above, a number of hypotheses were generated in Year 11 
-and tested in Year III. A detailed discussion of the evidence for and 
against each hypothesis Is presented In Interim Report V II Ji/o1 ume 1 . 
The analysis of determinants of implementation was organized into four 
areas. The rationale for considering each of these areas Is presented 
here, and a summary of the factors found to be most important Is presented 
in Table 3. 

The PDC: setti ng .^ The setting for PDC Is a potentially important 
determTnant since no change effort occurs within a vacuum^^exi sti ng 
regular! ties \have to^ be al tared, PDC, as a major attempt to modify existing 
Head Start and elementary school programs and the linkages between them, Is 
particularly susceptible to the Influence of existing attitudes, policies, 
and practices In the district and community. 
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Local initiation of PDC , Decisions to adopt large-scale federal 
pro^rarns, such as PDC, are to a large extent polttical acts. iHus, there 
Is an Initial "push" from the federal level to obtain the necessary humber 
eof adoptions. Often, local decisions to participate are based on decisions 
made at administrative levels above the Individual schools or centers 
that will ultimately be participating In the project. In the case of 
PDC, d^^ferent sites made their decisions to participate In PDC in 
different ways. The relationship between some of these procedures and 
later Implementation levels was Investigated. 

Planning year activities . Planning year activities were hypothesized 
to be major determrnants of later implementation. PDC has been unique 
among recent federal demonstration programs In that a full year of local 
planning was permitted before the program actually got underway. Many 
^project personnel who remember the difficulties encountered In attempting 
to set up Head Start or Follow Through programs in a matter of weeks 
applauded this feature of PDC. ACYF^s expectation was that fol lowl ng 
the planning year., the complete PDC project would be Implemented immediately 
with no sequencing either by component or by grade levels. The planning 
year and associated expectations thus created s I tuat 1 ons that could 
influence later implementation In important ways. 

f mplementat f on year strateg les , events ^ and activities . Finally , 
of course^, the act I vi t les and events of the Implementation period Itself 
would be expected to determine the implementation levels that were observed 
In our evaluation. Within the basic framework provided by the federal 
program guidelines, each site was encouraged to develop Its own unique 
program^ employing strategies considered appropriate and necessary by 
local staff. In practice^ sites diverged considerably In the! r approaches 
to PDC, .and these approaches were expected to directly affect the Imple- 
mentation levels that were later achieved. 

Tablets attempts to summarize the complex array of factors affecting 
Implementation by displaying the factors, events, or strategies that have 
had the greatest Influence on the Implementation of the PDC guidelines. 
The information 5s organized according to the four areas described above. 
The best Interpretation of this 1% that, when the experiences of all PDC 
sites were examined, these act ivl ties, or events were found to have Important 
Influences on ^the extent to which programs were able to implement the 
guidelines. Although It might seem that a successful si te would simply 
be the one with the most factors present, the reality Is somewhat more 
complex. The elements are not simply additive. Those listed on the left 
side of the table frequently constrained those to the right, so that if 
certain elements of the PDC setting were not present (such as a prior 
Head Start'-elementary school relationship), It was more difficult for the 
presence of an activity to the right to compensate. The Interaction of 
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Table 3 



Four Sati-of Factori, Events, and jtritsgiesjhit Contribytid to Ifnpleniintation During the First Three Years 



THE PDC SETTING 



Location In a niid-sized coMnity and 
ichooi district 

Soii prior relitionship between Haad 
Start and el«ntirj/ prografiis (joint 
idiiii'nlitrationj geographic proxiffiit^, 
$(Dphai1s 0 



PfHxistiiig district philosophies, 
priorities, ligislatlnn, prograoii that 
art sifnilar to or coptible with PDC 

At least a toiniiiial nuiiiber of miti^ 
cppynity resources 

High yoncentratlons of 'minority groups 
in the schooli or centers (affecting 
bilingual/iiitjlticylturil cooiponints only 

High conCintrition of Head Start grad- 
uitis in elidntiry clasiii ^ . 

High nyinber of mtioritiei In posltioni 
of authority {affecting bilinpl/ 
multicultyral cGmponents only) 

i 

Hlnoriry groups actively seeking to 
inaintiin their own language or cultural 
traditions (affecting bilingual/ 
fnulticultural cpftiponinti pniy) 

Favorable parental attitydes toward 
schools and fideril prograins and 
tfaditlons of parent Invoheniint in 
schools 



LOML INITIATION OF POC 



Broad participation by Head 
Start and alinientary per- 
ents, teachers, and adfuinii- 
trators In the first pro- 
posal writing 

School district designated 
the Head Start delegite 
agency (pnoiarily affecting 
the start-up period) 



PUiING yEAR ACTIVITIES 



Planning bigafi iirly 

TeacherSi parfints, adminlstritors 
Involved in planning 

High nufnbir of plinning tisks coinplited 

Teachirs pirticlpated voluntarily or - 
at leiEt given the option to trinifar 

Teachers epriinced in initryctional 
approaches analogpyi to PDC 

Coordinator with ixpiriince as in 
adninlitrator 

Hoitly younger tiaehers participating ■ 

tey staff iKperienced iii educationil 
change 

Key stiff fiHiliir with the workings of 
the school district ind Head Start 
prggrMs 

Parent Involviment coordinator frofn the 
coiiunlty and with profassionii epri* 
anci invoivlng parinti in schools or 
centiri, rather than aKperlenci only' 
as a volunteer 



IHPLEMEfJTATlOfI STWTEGIES, 
mi m ACTIVITIES 



PDC staff given line authoflty it both 
Head Start and eleiintary levels ' 

PDC staff houiid eithir with the glann- 
tary teichers or with school djitrict 
stiff 

PDC has full and active lupport from 
diitrict officlilSi Head Start centir 
director, principal 

Copnenti aisigned to specific Individ " 
dyali . 

No individusl has Tesponsibility for 
mors than two componants 

CofSponint reiponsibilities span both 
the Head Start and elefintary leiTT ■ 

Procedyrii for regular and frequent . 
coiiynieation are foniially establisliai 

Hiny participants involved in planning 
ictivitiis 

Project has had continuity of staffing 

Sites developed ^r pyrchased cyrricyluHi/ 
or willingly kept exiiting curriculur 

Sites have had a large araount of^training 
for participants 



, factors worked in the oppos! te way as well . In some cases the presence 
of one factor more than compensated for the absence of another (e,g,j 
if participation in proposal writing had not been broad, an effactive 
coordinator could r|ake up for this by' careful ly iricludlng the elementary 
school principal aiid Head Start director in the PDC communication net" 
work). In the final chapter of this report, we present our synthesis 
of this Information in eight general conclusions about POC Implementat lon-^-^our 
attempt to describe the "lessons learned'* about the program,^ 



Cost Study 

An accurate estimate of what it might cost to replicate PDC in similar 
communities throughout the country is Important for ACYF's planning. To 
estimate these costs, an approach was designed by Development Associates 
(DA) to ascertain the value of' program resources, rather than the "cost-' 
of PDC. These resources include not only the specific PDC grants, but 
also the respurces of the Head Start programs ahd elementary schools 
involved in the program. PDC is also responsible for utilizing community 
resources In support of program goalis. Thus, a number of public social 
service agencies and individuals such as doctors, dentists, parents, and 
business leaders may function as program resources, and the value of 
their services (even when donated to the program) was Included as part 
of the value of the programs. 

. With this approach, the dollar value of all goods and services 
associated with PDC will appear high. It must be remembered that these 
figures do not represent the cost to the government of operating the 
dembnstratipn program; rather, the figures represent what the cost would 
be I f orie7were to purchase allifelements that combine to yield the total 
program effort. Since the demonstration program depends so heavily on 
resources already ayillable through Head Start, the public schools, 
and the community, this Cost Study approach is referred to as a study of 
resource utilization, and the dollar figures reported are referred to, 
not as costs, -but as value of resources ut i 1 i zed . 



iThis first phase of the PDC evaluation leaves unanswered questions 
about relationships between program implementation and Impact on 
children. Preliminary analyses to address this Issue are reported 
in Interim Report VI i, Volume 3» but are too exploratory to yield 
clear conclusions at this stage. 



Development of a Cost Accounting System 

To determine the value of the respurces used and to allocate these 
resources across program components, a comprehensive cost accounting 
system was designed during Year I and refined for Years I I and III. 
In order. to ensure consistency of data across sites, standard def I n I t Ions 
were extracted from the PDC guidelines and standardized procedures and 
Instruments for recording and assigning values to various resources were 
established. No cost data were collected during the first program year 
since that year was used for planning and no children were served. 
During the implementation of the system in Year II, OA's cost specialists 
made two visits to each site to familiarise PDC staff with the definitions 
instruments and procadures/ Three requests were made of PDC staff. 
The, first was to record each PDC grant eKpendlture and specify the program 
component for which it was spent. The second was to record each non-- 
cash contribution made, to the PDC effoVt, The third request was for 
each PDC staff member' to maintain a record during one week each quarter 
to indicate how much and what percentage of their time was spent on 
work related to each PDC program component. 

Data were collected on a quarterly basis In order to provide an 
opportunity to monitor the quality of the Incoming Information* During 
site visits at the end of each year, cost specialists determined the 
dollar value of the resources utilized. Where actual dollar values 
were not recorded, estimates from local sources were obtained. For 
services contributed, the actual or estimated time each resource was 
used was also required. 



Year i I I Data Col lection 

». 

For the third program year the Cost Study was expanded to include 
the determination of resources utilized at comparison programs and to 
calculate the value of these resources. This involved conducting Inter- 
views with Head Start center directors, elementary school principals, and 
their respect ive .administrative support staffs. In addition, salary 
information was obtained for all personnel who worked directly with or 
in support of the Comparison programs. For these sources, the resources 
utilized and costs Incurred were tabulated and categorized Into the seven 
PDC program components. The data from both the PDC program and the 
comparison programs were then compiled and comparisons were made of .the 
similarities and differences In resource utilization patterns. Findings 
were presented In Interim Report VIII, and are surrwflarlzed here. 



Third Year Cost Findings 



During the third PDC program year the total value of resources 
utilized by 12 PDC programs was $7,^32,076^" Of this total, 16.9 
percent was derived from the POC grants, 56*5 percent from school districts, 
10.5 percent from Head Start, 10*8 percent from other federal resources, 
and 5.3 percent from local community sources. A comparison of the percentage 
distributions of the Early Childhood Schools (ECS) programs and the 
Preschool -School Linkages (PSL) programs revealed that the largest difference . 
between the two nrodels was In the contributions made by school dtstrlcts--60.8 
percent. of the resources utilized by PSL sites came from the school districts 
while only ^9.8 percent of the resources utilized by ECS sites came from 
the school districts; A difference in the proportion of the PDC grant 
contributjon was also found between the two PDC model s-^-Zl . 7 percent 
of the resources utilized by ECS sites came from the PDC grants, whereas 
only 13*8 percent of the resources utilized by PSL, sites came from the 
PDC grants. Differences between the two models in contributions made by 
the other sources were small. 

Thevalue of the total resources was highly related to the number ^ 
of children served by the program, which In turn, was related to the 
number of elementary schools Involved with PDC. PSL programs had an 
enrollment almost twice as large as the ECS programs (4,519 students 
In PSL programs vs. 2,222 in ECS programs) . The percentage of the PDC 
grant in relation to total program resources varied from 7.0 percent 
at One site to 28.6 percent at another. This variability appeared to be 
a function of the availability of resources In each community. 

Education received the greatest share of program resources by far. 
Over the 12 PDC programs, 65.2 percent of the resources were utilized' 
for the education component, 10.6 percent for administration, 9.6 percent 
for services to the handicapped and learning disabled, 7^0 percent for 
developmental support services, 3*9 percent for parent involvement , 2.2 
percent for bi 1 Inguar blcul tural and mul tlcultural activities^ and 1.5 
percent for training. The only real difference between program modey^ 
in this distribution was in administration: ECS programs utilized IJuk^ 
percent of their resources for administration whereas PSL programs utilized 
8.8 percent. When PDC resources were analyzed by type of expenditure 
it was found that gOTS^p^rcent was utilized for personnel, 3^7 percent for 
facilities, 3.0 percent for contractual services, 1.4 percent for material s , 
and 1.2 percent for travel and transportation* These distributions of 
resources by type and by components were very similar for Years 1 1 and III, 
indicating that the overall program emphasis in terms of expenditures 
and resource utilization remained about the same from one year to the next. 



iBecause the Arizona project was so different from the other sites in 
terms of factors affecting costs, It was not Included In the Cost Study. 



ERIC 



Across all 12 programs the average value of total resources utilized 
was $1,102 per child* The value for ECS programs was $1,309, per childp 
the value for PSL programs was $1,001 per child. Since the value of 
the PDC program grant averaged only $192 per childj these figures 
illustrate the very great extent to which additional resources have been'^ 
utilized by the programs* 

The collection of data from the comparison Head Start centers and 
schools showed that the difference between PDC and comparison programs 
in the value of total resources utilized per child was $192, or approxi-^ 
mately equal to the amount of the PDC grant per child* It was also found 
that the distribution of resources across program components was generally 
comparable. There were slight differences in the' proportion of resdurces 
allocated to some areas, which reflect the emphasis of , PDC In providing 
a more comprehensive program. Whereas- the comparison program data showed 
a higher proportion of .resources allocated to the education areas (an 
average of 71 percent vs. 65 percent for the PDC program) , the PDC 
programs distributed their resources so that greater proportions were 
allocated to parent involvement and developmental support services. 
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IV 



CONCLUSIONS AND IMPLICATIONS 

». ' • 

This phase of the PDC evaluation has been designed primarily to 
determine wfrether conditions are suitable for proceeding with a longitudinal 
study of PDC's process and effects through thkird grade. 'At the same trme, 
thei^evaluatlon has examined the Ifuplementation process during the" first 
three years and provided an^assessment of program costs. 

Previous reports ii^ thbs series have indicated that there are 
suitable comparison groups at the POC sites; that the child measures - ^ 
are sufficiently rej iable, valid,, and sensitive to change; and that sample 
sizes are adequate to withstand the effects of attrition, Ja the past 
year, reports have examl^d the effects of PDC on chrldren during the 
Head Star.t. year and^e>6prored factors affectJng program Implementation, ^ 
These two'aspects of the findings* are summarized here. 



^ i The Impact of PDC on Children's Development 

The first group of children to be evaluated entered PDC Head Start 
centers in faTJ, 1976* ^ Impact measures, which include a variety of tests, 
observations, ^n^ ratings, were -administered In the fall and again |n ^the 
spring of^the Head Start year^ and PDC children's progress was compared 
with the progress of similar children in nearby non^PDC Head Start centers. 
Although 'a few small differences were found between the groups, overall 
the findings confirmed the expectation that 4he two groups would show 
about the same degree of progress .through the Head Start year. Since 
PDC Is designed to provide greater coatlnuity In the transition from Head 
Start to elementary ^chool , \t is reMonable to expect future testing to 
show PDC*s real impa'c-t. « 

After looking for^overall PDC fffects, the poss i bi 1 1 ty of effects 
under specffic conditions was investigated*. For examplej analyses eKamined 
whether PDC mlght^ have different effects^ for boys than for girlSj or'^mighf 
produce greater gaSns In one site than in another. No d i f ferences In 
the program's effi^css under these different conditions were, fojurid, heading 
to the conclusion that, through the Head Starts year at least, PDC is 
equal ly ef^fect I ve for boys and girls^ for all ethnic groups, for children 
from air sites, 'and for children ^yho^did or did not have prior preschool 
exper i ehce . V > - 



Th&--.^f indings of the first phase of the Impact Study can be summarized 
as f o 1 1 ovvs : 

# Conditions are suitable for a longitudinal study of , 
PDC's impact on children through third grade. 

m At the end of ^3 year of Head Start, children in PDC and 
comparison groups showed essentially the same degree of 
progress, as expected. 

# Progress of the two,, groups was equal not just in general, 
but also Under all the special conditions examined/ 

} At this stage, the most important contribution of the PDC evaluation 
has been the knowledge gained about program Implementation. The imple^ 
mentation Study, as summarized next, has prov 1 ded extens I ve information 
about what has been accomplished In the program's initial three years 
and about some of the reasons for those accomplishments. 



Lessons Learned About PDC I mp 1 ementarl on 

In 1976-77 PDC Implementation acti vi t les were intensively studied 
at nine s 1 tes^^prof I les of program Implementation were developed and 
factors shaping implementation were analyzed* The .measurement of imple^ 
mentation indicated that almost all sites received high ratiags In the 
education component p particularly In areas concerned with development 
and implementation of a curriculum and diagnostic system. In contrast, 
ratings in other components--and especially in parent I nvol vement-^were . 
highly variable from site to. site. 

General Cone] us ions . 

On the basis o.f these analyses, eight general conclusions about PDC 
Implementation were drawn :^ 

m No single factor^^ event was sufficient to -'make'- or "break" 
a project; only combinations of factors operated to Influence 
' } mp 1 emen ta t i on . 

# The single most powerful set of determinants^ of implementation 
during the first three-years was the educational and community 
setting for each project* 



# The second most important set of determinants of implementat ion 
was the background, creativity, and Initiative of PDC staffs. 

m In general. Early Childhood School sites were able to implement 
the PDC guidelines more readily during the early years than 
were Preschool ^School Ll^nkage sites* although this does not 
mean that the potential for ultimate success is any greater 
for one model than for the other. 

m The planning year was a critical factor in the Implementation 
of PDC. 

0 When some form of planned sequencing of Implementation was 
adopted j sites made more rapid progress In their areas of 
focus. 

# Implementation proceeded most rapidly where administrative 
legitimacy for PDC staff had been established at both the 
Head Start and elementary levels. 

# Implementation proceeded most rapidly where a sense of 
"ownership" of PDC had been established among staff at 
both the Head Start and elementary school levels. 

- - ^ - . ■ 

Some Specific Factors' that Affected PDC Implementation 

In addition to these general conclusions, a number of specific 
factors emerged as most clearly related to program implementation durin 
the initial years. They are listed here in four areas^^the PDC setting 
local initiation of PDC^ planning year activities, and implementation 
activities during the implementation year. 

The PDC setting . Five factors associated with the setting of the 
local PDC program seemed to be associated with higher Implementation 
1 eve Is: 

*■ 

m Location in a mid-sized community (20,000^100,000 population) 
and mid-sized school district; 

m History ofi close cooperation or Joint administration of 
Head Start and elementary school programs by the school 
'district; 

m Pre^eKlstlnq district philosophies, priorities, legislation, 
and programs similar to or compatible with PDC; 

r 
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• High concentration of the target ethnic groups In the schools 
or centers (affecting Implementation in the bslinguaN 
multicultural component only); 

• Favorable parental attitudes toward schools and federal 
programs . 

Local initiation of PDC , Two features of the situation surrounding 
the beginnings of PDC seemed to confer at least an initial advantage 
because their presence meant that certain Important linkages demanded by 
PDC were already in place as the tasks of planning and implementation began: 

& Broad participation by parents, teachers and administrators In 
the first proposal writing (coordination at this stage between 
the Head Start center directors and elementary school principals 
who would later participate in the project was particularly valuable) 

• Oesignation of the school district as the delegate agency for the 
Head Start program, and thus for PDC (this Increased the likelihood 
that Head Start and the schools had already established a working rel 
tionship prior to PDC, it made it easier for PDC staff to have formal 
authority that spanned both levels, and It made It easier for the 
elementary school teachers to accept the project). 

Planning year activities . Although all programs were granted this 
special perTod, they^did not'use it equally effectively. Five features 
of planning year activities stood out as contributing to higher levels 
of Implementation in the third year^ 

• Involvement of teachers, parents, and administrators in 
p 1 ann i ng ; 

m Voluntary participation of teachers; 

• Coordinator experienced as an administrator; 

m Key staff members familiar with the workings of the school 
district and Head Start programs; 

^ • Parent involvement coordinator from the community and with 

professional experience Involving parents in schools or centers. 

implementation strategies, events, and act i vl t i es . I mp 1 emen ta 1 1 on 
activities during the implementation year were also important^ to the 
success of PDC. Higher levels of Implementation .were associated with 
the following conditions^ 

• Clearly delineated lines of authority for PDC stafi at both 
Head Start and elementary levels; 
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# PDC staff housed either with the elementary teachers or with 
school district staff; 

m Full and active support for PDC from district officials, 

the Head Start center director, and the school pr IncI pal (s) ; 

m Respcnsi bl 1 Ity for components assigned to specific Individuals; 

& No individual responsible for more than two components; 

m Component respons 1 b 1 11 t ies that spanned both the Head Start 
and elementary school levels; 

m Adoption of an eKlsting curriculum or purchase of an intact 
curriculum; 

m Frequent training for PDC participants. 



A Final Comment 



Broad Implications for federal effort^s to promote Innovative educa- 
tional programs have emerged from three years of Project Developmental 
Continuity, The approach ACYF adopted for PDC was one of providing a 
"framework for Innovation'' rather than dictating spec! fic innovative 
practices. Within this framework a number of strong local programs have 
developed. From the perspective of extensive implementation data, it 
seems that the PDC framework offers a potentially effective model of 
educational change. As the evolution of PDC continues over the next few 
years, the models for continuity should become stronger and clearer. 
PDC Is certainly altering the character of educational settings; the 
Importance of this altered character for the educational progress of 
children will become clearer as the demonstration program proceeds through 
the coming years . 
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IN THE PDC EVALUATION, 1975^1977 
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SUMMARY OF MEASURES USED IN THE PDC EVALUATtON, 19/5-1S77 



Cognitiv e-Language Measures 

Bilingual Syntax Measure (Burt; Dulay and Hernandez^Ch. , 1975) > ^ 
This test Is' des I gned to measure children's oral proficiency In English 
and/or Spanish grammatical structures. Simple questions are used with 
cartoon^type colored pictures to provide a conversational setting for 
eliciting natural speech. An analysis of the child's responses yields a 
numerical indicator and a qualitative description of the child's structural 
language proficiency In standard English or standard Spanish, Responses are 
written down verbatim, 

Wechsler Preschool and Primary Scale of Intel 1 igence, Block Design^ 
subtest (Wechsler , 1967)^" The task requires reproducing (constructing) 
designs wi th flat colored blocks, either from the examiner's model or 
from a picture on a' card. The measure taps problem-solving abilities, 
flexibility of response style, visual-motor organization, and execution, 

Wechsler Intel 1 Igence Scale for Children (WISC), Block Design su btest 
(Wechsler, 19^9) ^^ Similar to the WPPSI in that the child tries to make 
designs wi th' colored cubes to match designs made by the Interviewer or in a 
picture on a card, but suitable for older children. 

Conceptual Groupi ng (McCarthy Scales) .^ The child's abilities to deal 
with objects and "categories are tapped In this subtest. The child uses 
blocks and Is asked to respond to questions by choosing blocks that belong 
in a group, or by putting together groups (for example, "Find the square 
one," "Find all the big yellow ones," and so on). 

Draw-A-Chlld (McCar.thy Scales) . Child draws a picture of a child 
of the same sex. Scoring credits children for their ability to articulate 
parts of the body in their drawings. 



ifiurt, M., Dulay, H., S Hernandez-Chavez, E, B11 ingual Syntax Measure. 
New York: Harcourt, Brace, Jovanovich, 1975. 

^Wechsler, D= Wechsler Pres chool and Primary Scale of Intel 1 Igence: Manual . 
New York: Psychological Corporation, 1967. 

3Wechsler, D. Wechsler intelligence Scale for Children^ ManuaK New York: 
Psychological Corporation, 19^9. ~ 

^McCarthy, D. McCarthy Scales of Children's Abilities: Manual , New York: 
Psychological Corporation, 1972. 
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Say and Tel 1 (C! RCUS) . ^ This test is designed to assess descriptive 
ianguage'abll I ties. "The child is given objects (a pencil and two pennies) 
to talk about and describe to the interviewer. The scoring is based on 
categories of attributes which the child mentions. 

Verbal Me mory (HcCarthy Scales). Two perspectives on short-term memory 
are obtained. The"chlld Is asked to repeat sequences of words (Verbal 
Hemory-l) and to repeat or retell as much as possible of a one paragraph 
story (Verbal Memory^3) * 

Verbal Flu ency (McCarthy Sca les), AbMity to recall information by 
conceptua'l categories Is mMSured by this test. The child is asked to name 
as many members of specific catsgories (e.g., animals) as he/she can. 



Socj a I -Emot I onal Measu res 

PDC Child Ra ting Sca le (High/Scope FQundation, unpublished) . /This 
instrument Ts'des I gned as ~a measure of social competence to be administered 
by the respective classroom teachers of the children rated. For each of the 
39 items, specific behaviors such as '-Uses words or wits to influence 
others'' are rated on a S-point scale according to frequency of occurrence 
("Very frequently'' to "Rarely^'). Six scales were derived thrgugh factor 
analysis: ^friendliness, aggressi veness , , perseverence , independence, 
sel f ^assurance , and resourcef ul ness . 

PDC Classroom Qbservation System (High/Sco p e Foundat ion , unpubli shed) . 
The PDC observation system was developed to provide information about 
children's classroom behavior along dimensions pertinent to the social- 
emotional goals of Project Developmental Continuity. The system focuses on 
aspects of an individual child's behavior, verbal or non-verbaV, that reflect 
the child's attitude toward himself, and on the child's social competence as 
demonstrated in his interaction with peers and adults. 

Using a time-sampling method, trained observers observe each child for 
five minutes at two different times during the day and code the! r behavior , 
Into four general categories: "nonlnvol ved,'' ''involved,'' ''Interacts with 
peer," and "Interacts with a:!ult;'' A fifth category, "act ivi ty level 
is included to provide information concerning the context in which these 
behaviors were observed. Each of these categories includes subcategories 
that are designed to identify the frequency and nature of specific behaviors 
within the general category. 



ICIRCUSV Educational Testing Serv i ce . Pr i nceton , New Jersey, 197^. 
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PreschoQi Interpersonal Problem-^Solving Test (Shure and Spivack, 
1974).^ The PIPS attempts to assess the child's ability to name 
alternative solutions to a life-related problem--that of obtaining a 
toy from another child- Paper cut-outs of boys, girls and toys are used 
In presenting the problem. Arrong inner-city four^year-^ol ds attending the 
Philadelphia Get Set day care program, those judged as better adjusted 
by their teachers were able to conceptualize a greater number and a wider 
range of alternative solutions to real-life problems than were their more 
poorly adjusted classmates. 

Pupil Obse rvation Checklist (H 1 gh/Scope Foundat i on , unpublished) , 
This is a rating scaTe cons 1 St 1 ng of twel ve 7-^pot nt bipolar adject[ves 
derived from a similar scale used In the Home Start evaluation. ^ The 
tester rates each child using this instrument after he or she has administered 
all the other measures In the battery to the child. Scores are derived for 
two scales: task orientation and sociability. 

Stephens-Delys Reinforcement Con t I ngency ! n terv i ew (Stephens and Delys, 
1972)7^ thiV measure finds out whether a child thinks that his own behavior 
would' change what other people do or feel. In other words, If a child changes 
his behavior, will his teachersS parents' or friends' attitudes or behaviors 
change as a result? Twel ve ^quest Ions were usedj for example, ''What makes 
your teacher happy?". 



Psychomotor Measures 

Arm Coordination (McCarthy Scales). This measure assesses a child's 
arm coordination. Activities Include ball bouncing, catching a beanbag, 
and throwing a beanbag at a target* 

Block Bu ilding (McCarthy Scales). This interview Is designed to assess 
a child's Tlnger and harTd coordination and his perceptions of things and 
spaces as he builds things out of blocks to copy what the Interviewer has 
bul 1 t. 



^hure, H. B., & Spivack, G. The P I PS Test Manual , Philadelphia^ Hahneman 
Medical College, 197^. 

^"Love, J.» et aK National Home Start Evaluation Interim Report VI I . 
YpsllantI, Hich~^ High/Scope Foundation, March 1976^ 

^Stephens, M. W. , & Delys, P. Stephens-De lys Re i nforcemen L ^contingency 
Interview. Purdue University, 1972. 
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Leg Coord t nation (McCarthy Scales) . This task measures a child's leg 
coordination^ Activities Include walking backwards, walking on tiptoes, 
and skipping, 

Site-Specific Measures 

Do You Know,..? (CIRCUS) . This Is a general information interview* 
The child chooses appropriate pictures which answer the fnterviewer's question. 
This task taps the ch i 1 d ' s exper 1 ence in a variety of areas (health, safety, 
social standards, consumer concepts). 

Opposite Analogies (McCarthy Scales) . In this test of classification 
skills, the child is asked to supply the rnissing word in an analogy (for 
example, -The sun is hot; ice is - 

Wepman Aud itor y D j scr im 1 nat i on Test (Wepman, 1973).^ This measure 
assesses the child ' s abi 1 i ty to tell the difference "between sounds. 



Other Measures 

Adult Language Check . This measure is used in the bi lingual 
blcultu^ral demonstration sites to obtain an indication of the languages 
the adults in the classroom use during their Interactions with children. 
The Interviewer sits in the classroom for a two-hour period and records the 
language used by the teachers and aides within five-minute Intervals. 

Height and Weight . All children are weighed and measured during the 
same two-week period at testing time. 

'PD C Faces Interview (High/Scope Foundation, unpublis hed). This test 
Is des]g7ied to assess the child 's attitude toward school and his teacher. 
The child Is asked to point to one of five faces (which range frdm happy to 
sad) as he is asked questions about school and his teacher. 

Preschool Productive Language Asse ssment Task (PP LAT) , (High/Scope 
Foundat ion > unpubl i shed) . This measure Is designed to assess oral language 
proncTency. Each child is provided with a variety of materials and Interacts 
Informal lywl th the adult tester for about thirty minutes. The entire session 
is tape-recorded and later transcribed for coding. 



^Wepman, J. Auditory Discrimination Test . Chicago^ Language Research - 
Association, Inc., 1973. 
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